Files
Introduction
What is it?
@workertown/files
provides simple REST API for implementing a file storage system, with support for "signed" public uploads.
You store a file (blob of data) at a path, and can then retrieve it / update it / delete it from said path.
Getting started
Installation
You can install @workertown/files
via npm
/yarn
/pnpm
:
npm install @workertown/files
Creating a server
In your main file (e.g. worker.ts
), import the files
factory function and call it.
import { files } from "@workertown/files";
//...or `import files from "@workertown/files";`
const server = files();
//...probably `export default server;`
Like all Workertown services, the created server (based on Hono) instance with a fetch
method.
The server
function accepts a single argument, an optional options object. This options object allows you to customise the files service to fit your needs (see configuration for a full set of options).
import { files } from "@workertown/files";
// These are the default values...
const server = files({
auth: {
apiKey: {
env: {
apiKey: "FILES_API_KEY", // Environment variable for the API key
},
},
basic: {
env: {
username: "FILES_USERNAME", // Environment variable for the admin username
password: "FILES_PASSWORD", // Environment variable for the admin password
},
},
jwt: {
env: {
jwksUrl: "FILES_JWKS_URL", // Environment variable for the JWKS URL
secret: "FILES_JWT_SECRET", // Environment variable for the fixed JWT secret
issuer: "FILES_JWT_ISSUER", // Environment variable for the JWT issuer
audience: "FILES_JWT_AUDIENCE", // Environment variable for the JWT audience
},
},
}, // See the "Authentication" section for all of the available options in `auth`
basePath: "/", // Base path for the server to serve endpoints from
endpoints: {
v1: {
admin: "/admin", // Base path for the server to serve admin endpoints from
files: "/v1/files", // Base path for the server to serve files endpoints from
uploads: "/v1/uploads", // Base path for the server to serve uploads endpoints from
},
public: "/", // Base path for the server to serve public endpoints from
},
env: {
cache: "FILES_CACHE", // Environment variable for the cache KV binding (Cloudflare Workers only)
database: "FILES_DB", // Environment variable for the D1 database binding (Cloudflare Workers only)
files: "FILES_FILES" // Environment variable for the R2 binding (Cloudflare Workers only)
},
});
//...
Concepts
Files
@workertown/files
stores files against a unique path via a secure REST API, with optional support to allow public uploads. A file is essentially any blob of data.
Uploads
An upload is a unique identifier for an upload operation that you can create in your secure server environment and then issue to a public client to allow public uploads.
An upload is identified by an id
(a unique string
identifier), and contains a path
(the path to store the file at), and optionally a callbackUrl
(the URL to call when the upload is complete) and metadata
(any additional JSON
data you want to be associated with the file).
Here is an example of an upload represented in JSON
:
{
"id": "b6628020-e402-4e45-9102-d5cba9887010",
"path": "/test/file.txt",
"callbackUrl": "https://example.com",
"metadata": {
"test": true
}
}
On a successful upload, is there is a callbackUrl
, it will be called with a POST
request with the body of the request being the JSON
representation of the upload (as above).
To verify that the upload is valid, the callbackUrl
will be signed with an X-Workertown-Signature
header that contains an HMAC of the upload's JSON
representation, using a configurable secret (see configuration for more details).
Here is some example code to verify the signature in Typescript:
const secret = "<SOME SECRET>";
const encoder = new TextEncoder();
const key = await crypto.subtle.importKey(
"raw",
encoder.encode(secret),
"HMAC",
false,
["verify"],
);
const valid = await crypto.subtle.verify(
"HMAC",
key,
encoder.encode("<`X-Workertown-Signature` HEADER VALUE>"),
encoder.encode(JSON.stringify({ /* ...upload JSON */ })),
);
How does it work?
File storage at the edge
@workertown/files
is a REST API abstraction on top of your existing object storage solution. It provides a simple interface for storing files at a path, and retrieving them. As most object storage solutions are already designed to be HTTP compatible (and therefore edge compatible), @workertown/files
aims to simplify the interface to your file storage.
What are the limitations?
At the edge, memory is a limited resource. The files you upload are buffered into memory fully before upload - this means that if you intend to have files larger than 120MB or so in size, you should consider using your object storage solution directly, and not via @workertown/files
.
The "EJECT" button
Things don't always work out.. and software doesn't always scale with your business, or stand against the general test of time. That's OK - it's actually a good thing (mostly)!
If you've been running @workertown/files
in production, there isn't much to do after you've deleted your files service - your files are stored directly in your object storage of choice, and therefore they have no direct dependency on @workertown/files
.