File Storage

What is object storage?

Object storage provides a simple way to store large quantities of files. These services are very secure, especially when setup with correct permissions. These services will also encrypt and secure your data.

Security

This may sound obvious, but always make use of IAM users and don't give people access to your bucket. For all object storage providers, make sure your buckets are private. No one, under any circumstances, should be able to list contents of your bucket. When you click on a file within Hudu, we create an expirable link that will only allow you to view the file for a very short amount of time.

Options for Object Storage

Other object storage providers are not tested. However, they should work as long as they are S3-compatible.

Testing

In order to test that your object storage is working, go to your user settings (found at the bottom of the sidebar when you are logged in) and try and add a profile photo. If the photo doesn't appear (e.g. broken link), then something went wrong. It may be helpful to look in the developer tools of the browser you are currently using. If you see an error, it may because your bucket url is incorrect, or you are trying to request from a non https source.

Examples .env file configurations for each provider:

No object storage

To use local storage, make sure that the following lines are placed in the docker-compose.yml file:

- app_data:/var/www/hudu2/public/uploads/

- app_data:/var/www/hudu2/uploads/

under the volumes section of both the worker and app sections.

and then set your .env to look like:

  USE_LOCAL_FILESYSTEM=true
  AUTHENTICATE_UPLOADS=true
  

We recommend testing local file storage by uploading some test files, and then re-running

sudo docker-compose down && sudo docker-compose up -d
and checking on the files to test that all file storage locations are mapped correctly.

Amazon S3. Leave endpoint blank.

  USE_LOCAL_FILESYSTEM=false
  S3_ENDPOINT=
  S3_BUCKET=bucket_name_here
  S3_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX
  S3_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
  S3_REGION=us-east-1
  

DigitalOcean Spaces. Fill in endpoint. Make sure endpoint has https://. Endpoint subdomain should match region. How can I create an API key?

  USE_LOCAL_FILESYSTEM=false
  S3_ENDPOINT=https://nyc3.digitaloceanspaces.com
  S3_BUCKET=bucket_name_here
  S3_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX
  S3_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
  S3_REGION=nyc3
  

Wasabi. Fill in endpoint. Make sure endpoint has https://. Endpoint subdomain should match region.

  USE_LOCAL_FILESYSTEM=false
  S3_ENDPOINT=https://s3.us-east-2.wasabisys.com
  S3_BUCKET=bucket_name_here
  S3_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX
  S3_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
  S3_REGION=us-east-2
  

Scaleway. Fill in endpoint. Make sure endpoint has https://. Endpoint subdomain should match region.

  S3_ENDPOINT=https://s3.fr-par.scw.cloud
  S3_BUCKET=bucket_name_here
  S3_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX
  S3_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
  S3_REGION=fr-par
  S3_REMOVE_ENCRYPTION_HEADER=true
  

Self-hosted Object Storage

We recommend using a cloud provider for storage. Storage is one of the harder parts to host yourself, and cloud providers make it easy and cheap to use their services.

We recommend hosting min.io on another machine (not the one Hudu is running on!)

Min.io. Make sure you have a running Min.io with a bucket already created, matching the one below. Note that there are two additional variables.

  USE_LOCAL_FILESYSTEM=false
  S3_ENDPOINT=https://minio.example.com:9000
  S3_BUCKET=testbucket
  S3_ACCESS_KEY_ID=minio
  S3_SECRET_ACCESS_KEY=miniostorage
  S3_REGION=us-east-1
  S3_REMOVE_ENCRYPTION_HEADER=true
  S3_FORCE_PATH_STYLE=true
  

Here is an article outlining how to setup Min.io on a Ubuntu 18.04 server with a SSL certificate

Notes on Min.io

Make sure the bucket already exists in Min.io that you are trying to connect to.

You can test Min.io by installing aws-cli (here is a guide) on the server running the Hudu application and seeing if you can connect.