Skip to main content

Configuring FileStorage adapters

MemoryFileStorageAdapter​

To use the MemoryFileStorageAdapter you only need to create instance of it:

./samples/memory-file-storage-adapter.ts
import { MemoryFileStorageAdapter } from "eridu-tech/file-storage/memory-file-storage-adapter";

const memoryFileStorageAdapter = new MemoryFileStorageAdapter();

You can also provide an Map that will be used for storing the files in memory:

./samples/memory-file-storage-adapter-with-map.ts
import { MemoryFileStorageAdapter } from "eridu-tech/file-storage/memory-file-storage-adapter";

const map = new Map<any, any>();
const memoryFileStorageAdapter = new MemoryFileStorageAdapter(map);
info

MemoryFileStorageAdapter lets you test your app without external dependencies like @aws-sdk/client-s3, ideal for local development, unit tests, integration tests and fast E2E test for the backend application.

warning

Note this adapter doesnt have support for creating signed upload, signed download and public urls.

FsFileStorageAdapter​

To use the FsFileStorageAdapter you only need to create an instance of it, every setting is optional:

./samples/fs-file-storage-adapter.ts
import { FsFileStorageAdapter } from "eridu-tech/file-storage/fs-file-storage-adapter";

const fsFileStorageAdapter = new FsFileStorageAdapter({});

You can configure the root folder:

./samples/fs-file-storage-adapter-custom-location.ts
import { FsFileStorageAdapter } from "eridu-tech/file-storage/fs-file-storage-adapter";

const fsFileStorageAdapter = new FsFileStorageAdapter({
    location: "/my-custom-location",
});

You can configure codec used for file names:

./samples/fs-file-storage-adapter-with-codec.ts
import { Base64Codec } from "eridu-tech/codec/base-64-codec";
import { FsFileStorageAdapter } from "eridu-tech/file-storage/fs-file-storage-adapter";

const fsFileStorageAdapter = new FsFileStorageAdapter({
    codec: new Base64Codec(),
});
warning

Not encoding and decoding is required for FsFileStorageAdapter to maintain a flat hierarchy within the root folder and to ensure compatibility with OS-restricted characters.

warning

Not this adapter does not support signed upload, signed download and public urls. It also doesnt support explictly setting the content-type and it will instead infer the content-type from the file name.

S3FileStorageAdapter​

To use the S3FileStorageAdapter, you'll need to:

  1. Install the required dependency: @aws-sdk/client-s3 package:
./samples/s3-file-storage-adapter.ts
import { S3FileStorageAdapter } from "eridu-tech/file-storage/s3-file-storage-adapter";
import { S3Client } from "@aws-sdk/client-s3";

const s3Client = new S3Client({
    credentials: {
        accessKeyId: "AWS_ACCESS_KEY_ID",
        secretAccessKey: "AWS_SECRET_ACCESS_KEY",
    },
    region: "AWS_REGION",
});
const s3FileStorageAdapter = new S3FileStorageAdapter({
    client: s3Client,
    bucket: "bucket",
});

Other settings:

./samples/s3-file-storage-adapter-settings.ts
import {
    S3FileStorageAdapter,
    defaultPublicUrlGenerator,
} from "eridu-tech/file-storage/s3-file-storage-adapter";
import { S3Client } from "@aws-sdk/client-s3";

const s3Client = new S3Client({
    credentials: {
        accessKeyId: "AWS_ACCESS_KEY_ID",
        secretAccessKey: "AWS_SECRET_ACCESS_KEY",
    },
    region: "AWS_REGION",
});
const s3FileStorageAdapter = new S3FileStorageAdapter({
    client: s3Client,

    /**
     * The bucket option defines the S3 bucket to use for managing files.
     */
    bucket: "bucket",

    /**
     * The cdnUrl field can be used to define the base URL for generating public URL for a file. For example, If you use CloudFront alongside S3 to serve public files, the cdnUrl property should be the CloudFront URL.
     */
    cdnUrl: null,

    /**
     * Define ServerSideEncryption option for all objects uploaded to S3.
     */
    serverSideEncryption: "AES256",

    /**
     * Define a custom public url generator for creating public and signed URLs.
     */
    publicUrlGenerator: defaultPublicUrlGenerator,
});
info

Note this adapter with object storage services that are compatible with aws s3 like:

  • Cloudflare r2
  • Digital ocean spaces
  • Tigris
  • Supabase Storage
  • Minio

SftpFileStorageAdapter​

To use the SftpFileStorageAdapter, you'll need to:

  1. Install the required dependency: ssh2-sftp-client package.
  2. Create a pool of SFTP connections with sftpPool, which combines SftpFactory with a QueuePool.
./samples/sftp-file-storage-adapter.ts
import {
    SftpFileStorageAdapter,
    sftpPool,
} from "eridu-tech/file-storage/sftp-file-storage-adapter";

const sftpFileStorageAdapter = new SftpFileStorageAdapter({
    pool: sftpPool({
        connectionSettings: {
            host: "sftp.example.com",
            port: 22,
            username: "user",
            password: "password",
        },
    }),
});

Other settings:

./samples/sftp-file-storage-adapter-settings.ts
import { Base64Codec } from "eridu-tech/codec/base-64-codec";
import {
    PATH_TYPE,
    SftpFileStorageAdapter,
    sftpPool,
} from "eridu-tech/file-storage/sftp-file-storage-adapter";

const sftpFileStorageAdapter = new SftpFileStorageAdapter({
    pool: sftpPool({
        // Forwarded to client.connect()
        connectionSettings: {
            host: "sftp.example.com",
            port: 22,
            username: "user",
            password: "password",
            retries: 3,
        },
    }),

    // Writable root folder within the SFTP user's chroot
    location: "/uploads",
    delimiter: "/",
    codec: new Base64Codec(),
    pathType: PATH_TYPE.POSIX,
});

You can build the pool yourself, for example to change the pool size or to share it between adapters:

./samples/sftp-file-storage-adapter-custom-pool.ts
import {
    SftpFactory,
    SftpFileStorageAdapter,
} from "eridu-tech/file-storage/sftp-file-storage-adapter";
import { QueuePool } from "eridu-tech/generic-pool";

const sftpFileStorageAdapter = new SftpFileStorageAdapter({
    pool: new QueuePool(
        new SftpFactory({
            connectionSettings: {
                host: "sftp.example.com",
                port: 22,
                username: "user",
                password: "password",
            },
        }),
        {
            maxResources: 5,
            maxWaitSize: 100,
        },
    ),
});
info

SftpFactory connects a new SftpClient in create, checks idle connections with cwd() in validate, and ends them in destroy. Any IGenericPoolBase<SftpClient> can be passed as the pool setting.

warning

This adapter does not support signed upload, signed download and public urls. It also doesn't support explicitly setting the content-type and will instead infer the content-type from the file name.

FtpFileStorageAdapter​

To use the FtpFileStorageAdapter, you'll need to:

  1. Install the required dependency: basic-ftp package.
  2. Create a pool of FTP connections with ftpPool, which combines FtpFactory with a QueuePool.
./samples/ftp-file-storage-adapter.ts
import {
    FtpFileStorageAdapter,
    ftpPool,
} from "eridu-tech/file-storage/ftp-file-storage-adapter";

const ftpFileStorageAdapter = new FtpFileStorageAdapter({
    pool: ftpPool({
        accessSettings: {
            host: "ftp.example.com",
            port: 21,
            user: "user",
            password: "password",
            // Use FTPS over TLS
            secure: true,
        },
    }),
});

Other settings:

./samples/ftp-file-storage-adapter-settings.ts
import { Base64Codec } from "eridu-tech/codec/base-64-codec";
import {
    FtpFileStorageAdapter,
    PATH_TYPE,
    ftpPool,
} from "eridu-tech/file-storage/ftp-file-storage-adapter";

const ftpFileStorageAdapter = new FtpFileStorageAdapter({
    pool: ftpPool({
        // How the connection is opened and logged in
        accessSettings: {
            host: "ftp.example.com",
            port: 21,
            user: "user",
            password: "password",
            secure: true,
        },
        // Client-level options
        clientSettings: {
            allowSeparateTransferHost: false,
        },
        // Socket timeout in milliseconds, 0 disables it
        timeout: 30_000,
    }),

    // Writable root folder within the FTP user's chroot
    location: "/uploads",
    delimiter: "/",
    codec: new Base64Codec(),
    pathType: PATH_TYPE.POSIX,
});

You can build the pool yourself, for example to change the pool size or to share it between adapters:

./samples/ftp-file-storage-adapter-custom-pool.ts
import {
    FtpFactory,
    FtpFileStorageAdapter,
} from "eridu-tech/file-storage/ftp-file-storage-adapter";
import { QueuePool } from "eridu-tech/generic-pool";

const ftpFileStorageAdapter = new FtpFileStorageAdapter({
    pool: new QueuePool(
        new FtpFactory({
            accessSettings: {
                host: "ftp.example.com",
                port: 21,
                user: "user",
                password: "password",
                secure: true,
            },
        }),
        {
            maxResources: 5,
            maxWaitSize: 100,
        },
    ),
});
info

Set secure to true for explicit FTPS over TLS, or to "implicit" for legacy implicit FTPS. FtpFactory opens and logs in a Client in create, checks idle connections with pwd() in validate, and closes them in destroy. Any IGenericPoolBase<Client> can be passed as the pool setting.

warning

This adapter does not support signed upload, signed download and public urls. It also doesn't support explicitly setting the content-type and will instead infer the content-type from the file name.

WebdavFileStorageAdapter​

To use the WebdavFileStorageAdapter, you'll need to:

  1. Install the required dependency: webdav package.
  2. Create a client with createClient and pass it to the adapter.
./samples/webdav-file-storage-adapter.ts
import { WebdavFileStorageAdapter } from "eridu-tech/file-storage/webdav-file-storage-adapter";
import { createClient } from "webdav";

const webdavClient = createClient("https://webdav.example.com/files", {
    username: "user",
    password: "password",
});

const webdavFileStorageAdapter = new WebdavFileStorageAdapter({
    client: webdavClient,
});

Other settings:

./samples/webdav-file-storage-adapter-settings.ts
import { Base64Codec } from "eridu-tech/codec/base-64-codec";
import { WebdavFileStorageAdapter } from "eridu-tech/file-storage/webdav-file-storage-adapter";
import { createClient } from "webdav";

const webdavClient = createClient("https://webdav.example.com", {
    username: "user",
    password: "password",
    // Prefix for every path, when the server is mounted on a sub path
    remoteBasePath: "files",
    headers: {
        "X-Custom-Header": "value",
    },
});

const webdavFileStorageAdapter = new WebdavFileStorageAdapter({
    client: webdavClient,
    delimiter: "/",
    codec: new Base64Codec(),
});
info

The adapter keeps the WebDAV client it is given, so a single client serves every operation. Configure authentication, headers and remoteBasePath on the client itself.

warning

This adapter does not support signed upload, signed download and public urls. It also doesn't support explicitly setting the content-type and will instead infer the content-type from the file name.

SignedFileStorageAdapter​

The SignedFileStorageAdapter merges a regular file storage adapter with a URL adapter into a single ISignedFileStorageAdapter.

The adapter handles all file CRUD operations (create, read, update, delete, copy and move), while the urlAdapter handles public and signed URL generation.

This is useful for storage backends where file operations and signed URL generation are handled separately, for example a database-backed storage adapter combined with an S3 URL adapter. Since S3FileStorageAdapter already implements signed URL generation, you typically use SignedFileStorageAdapter when you need to pair a regular adapter (that does not generate URLs) with your own URL generation logic.

To use the SignedFileStorageAdapter you need to provide:

  • adapter: The underlying file storage adapter that handles file operations. It is not required to implement signed URL generation.
  • urlAdapter: A partial URL adapter for generating public and signed URLs. Only the URL methods your storage backend supports need to be provided.

Basic usage:

./samples/signed-file-storage-adapter.ts
import { SignedFileStorageAdapter } from "eridu-tech/file-storage/signed-file-storage-adapter";
import { MemoryFileStorageAdapter } from "eridu-tech/file-storage/memory-file-storage-adapter";

const signedFileStorageAdapter = new SignedFileStorageAdapter({
    adapter: new MemoryFileStorageAdapter(),
    urlAdapter: {},
});

You can provide the URL methods that your storage backend supports:

./samples/signed-file-storage-adapter-url-methods.ts
import { SignedFileStorageAdapter } from "eridu-tech/file-storage/signed-file-storage-adapter";
import { MemoryFileStorageAdapter } from "eridu-tech/file-storage/memory-file-storage-adapter";
import type {
    FileAdapterSignedDownloadUrlSettings,
    FileAdapterSignedUploadUrlSettings,
} from "eridu-tech/file-storage/contracts";

// The following helpers generate the signed URLs for your storage backend.
// Implement them using your storage provider SDK (e.g. AWS S3 presigned URLs).
async function generateSignedDownloadUrl(
    key: string,
    settings: FileAdapterSignedDownloadUrlSettings,
): Promise<string | null> {
    // ... generate the signed download URL
    return null;
}

async function generateSignedUploadUrl(
    key: string,
    settings: FileAdapterSignedUploadUrlSettings,
): Promise<string> {
    // ... generate the signed upload URL
    return "";
}

const signedFileStorageAdapter = new SignedFileStorageAdapter({
    adapter: new MemoryFileStorageAdapter(),
    urlAdapter: {
        async getPublicUrl(key: string): Promise<string | null> {
            return `https://cdn.example.com/${key}`;
        },
        async getSignedDownloadUrl(
            key: string,
            settings: FileAdapterSignedDownloadUrlSettings,
        ): Promise<string | null> {
            return generateSignedDownloadUrl(key, settings);
        },
        async getSignedUploadUrl(
            key: string,
            settings: FileAdapterSignedUploadUrlSettings,
        ): Promise<string> {
            return generateSignedUploadUrl(key, settings);
        },
    },
});
info

Any omitted URL method falls back to a no-op implementation:

  • getPublicUrl returns null
  • getSignedDownloadUrl returns null
  • getSignedUploadUrl returns an empty string

NoOpFileStorageAdapter​

The NoOpFileStorageAdapter is a no-operation implementation, it performs no actions when called:

./samples/no-op-file-storage-adapter.ts
import { NoOpFileStorageAdapter } from "eridu-tech/file-storage/no-op-file-storage-adapter";

const noOpFileStorageAdapter = new NoOpFileStorageAdapter();
info

The NoOpFileStorageAdapter is useful when you want to mock out or disable your FileStorage instance.

Further information​

For further information refer to eridu-tech/file-storage API docs.