Express File Uploads with Multer

Introduction

File upload is a common feature that almost every website needs. We will go through step-by-step on how to handle single and multiple files upload with Express, save it to the database (LokiJS), and retrieve the saved file for viewing.

The complete source code is available here: https://github.com/chybie/file-upload-express.

We will be using Typescript throughout this tutorial.

Install Required Dependencies

Run this command to install required dependencies

Run this for yarn:

yarn add express cors multer lokijs del

Or using npm:

npm install express cors multer lokijs del --save

• express: We will develop our API using ExpressJs
• cors: A node.js package that provides an Express/Connect middleware to enable Cross Origin Resource Sharing (CORS)
• multer: Node.js middleware for handling multipart/form-data.
• loki: LokiJS, a fast, in-memory document-oriented datastore for node.js, browser and cordova
• del: Delete files and folders

Development Dependencies

Since we are using Typescript, we need to install @types files in order to have auto-complete function (IntelliSense) during development.

Run this for yarn:

yarn add typescript @types/express @types/multer @types/lokijs @types/del --dev

Or using npm:

npm install typescript @types/express @types/multer @types/lokijs @types/del --save-dev

Setup

A couple of setup steps to go before we start.

TypeScript Configuration

Add a TypeScript configuration file. To know more about TypeScript configuration, visit the documentation.

{
    "compilerOptions": {
        "module": "commonjs",
        "moduleResolution": "node",
        "target": "es6",
        "noImplicitAny": false,
        "sourceMap": true,
        "outDir": "dist"
    }
}

• The compiled JavaScript code will be output to dist folder.
• Since Node.js 7.5+ support ES6 / 2015, we will set the target as es6.

Start Script

Add the following scripts.

{
    ...
    "scripts": {
        "prestart": "tsc",
        "start": "node dist/index.js"
    }
    ...
}

Later on, we can run yarn start or npm start to start our application.

1. When we run yarn start, it will trigger prestart script first. The command tsc will read the tsconfig.json file and compile all typescript files to javascript in dist folder.
2. Then, we will run the compiled index file dist/index.js.

Starting Express Server

Let’s start creating our Express server.

import * as express from 'express'
import * as multer from 'multer'
import * as cors from 'cors'
import * as fs from 'fs'
import * as path from 'path'
import * as Loki from 'lokijs'

// setup
const DB_NAME = 'db.json';
const COLLECTION_NAME = 'images';
const UPLOAD_PATH = 'uploads';
const upload = multer({ dest: `${UPLOAD_PATH}/` }); // multer configuration
const db = new Loki(`${UPLOAD_PATH}/${DB_NAME}`, { persistenceMethod: 'fs' });

// app
const app = express();
app.use(cors());

app.listen(3000, function () {
    console.log('listening on port 3000!');
});

The code is pretty expressive itself. We allow Cross-Origin Resource Sharing (CORS), set the connection port to 3000, and start the server.

Upload Single File

Let’s create our first route. We will create a route to allow users to upload their profile avatar.

Route

...
import {
    loadCollection
} from './utils';
...

app.post('/profile', upload.single('avatar'), async (req, res) => {
    try {
        const col = await loadCollection(COLLECTION_NAME, db);
        const data = col.insert(req.file);

        db.saveDatabase();
        res.send({ id: data.$loki, fileName: data.filename, originalName: data.originalname });
    } catch (err) {
        res.sendStatus(400);
    }
})

1. This is an HTTP POST function.
2. upload.single('avatar') is Multer middleware. It means we accept a single file with the field name avatar. File upload will be handled by Multer.
3. Multer will add a file property for request when it’s a single file upload.
4. We will then load the LokiJS collection/table(we will create this function next), and insert the request file req.file to the collection.

Load LokiJs Collection

A generic function to retrieve a LokiJS collection if exists, or create a new one if it doesn’t.

import * as del from 'del';
import * as Loki from 'lokijs';

const loadCollection = function (colName, db: Loki): Promise<LokiCollection<any>> {
    return new Promise(resolve => {
        db.loadDatabase({}, () => {
            const _collection = db.getCollection(colName) || db.addCollection(colName);
            resolve(_collection);
        })
    });
}

export { loadCollection }

Run Our Application

You may run the application with yarn start. I try to call the locahost:3000/profile API with (Postman)[https://www.getpostman.com/apps], a GUI application for API testing.

When I upload a file, you can see that a new file is created in uploads folder, and the database file db.json is created as well.

When I issue a call without passing in avatar, an error will be returned.

Filter File Type

We can handle file upload successfully now. Next, we need to limit the file type to image only. To do this, let’s create a filter function that will test the file extensions.

...

const imageFilter = function (req, file, cb) {
    // accept image only
    if (!file.originalname.match(/\.(jpg|jpeg|png|gif)$/)) {
        return cb(new Error('Only image files are allowed!'), false);
    }
    cb(null, true);
};

...

export { imageFilter, loadCollection }

Apply the Image Filter

We need to tell the Multer to apply our image filter function. Add it in upload variable.

...
import { imageFilter, loadCollection } from './utils';
...
// setup
...
const upload = multer({ dest: `${UPLOAD_PATH}/`, fileFilter: imageFilter }); // apply filter

...

Restart the application, try to upload a non-image file and you should get an error.

Upload Multiple Files

Let’s proceed to handle multiple files upload now. We will create a new route to allow user to upload their photos.

Route

...

app.post('/photos/upload', upload.array('photos', 12), async (req, res) => {
    try {
        const col = await loadCollection(COLLECTION_NAME, db)
        let data = [].concat(col.insert(req.files));

        db.saveDatabase();
        res.send(data.map(x => ({ id: x.$loki, fileName: x.filename, originalName: x.originalname })));
    } catch (err) {
        res.sendStatus(400);
    }
})

...

The code is similar to single file upload, except we accept a field photos instead of avatar, limit the total file to 12, accept an array of files as input, and reply result as array.

Retrieve List of Images

Next, create a route to retrieve all images.

...

app.get('/images', async (req, res) => {
    try {
        const col = await loadCollection(COLLECTION_NAME, db);
        res.send(col.data);
    } catch (err) {
        res.sendStatus(400);
    }
})

...

The code is super easy to understand.

Retrieve Image by Id

Next, create a route to retrieve an image by id.

...

app.get('/images/:id', async (req, res) => {
    try {
        const col = await loadCollection(COLLECTION_NAME, db);
        const result = col.get(req.params.id);

        if (!result) {
            res.sendStatus(404);
            return;
        };

        res.setHeader('Content-Type', result.mimetype);
        fs.createReadStream(path.join(UPLOAD_PATH, result.filename)).pipe(res);
    } catch (err) {
        res.sendStatus(400);
    }
})
...

1. We will return 404 if the image does not exist in the database.
2. We will stream the file as output, set the content-type correctly so our client or browser knows how to handle it.

Run the Application

Now restart the application, upload a couple of images, and retrieve it by id. You should see the image is return as image instead of a JSON object.

Clear All Data When Restart

Sometimes, you might want to clear all the images and database collection during development. Here’s a helper function to do so.

....

const cleanFolder = function (folderPath) {
    // delete files inside folder but not the folder itself
    del.sync([`${folderPath}/**`, `!${folderPath}`]);
};

...

export { imageFilter, loadCollection, cleanFolder }

Use it in our application.

...
import { imageFilter, loadCollection, cleanFolder } from './utils';
...

// setup
...

// optional: clean all data before start
cleanFolder(UPLOAD_PATH);

...

Conclusion

Handle file(s) upload with Express is easy with the help of Multer middleware. Multer is very flexible and configurable. You can go through the documentation and add more configuration as you need.

The complete source code is available here: https://github.com/chybie/file-upload-express.

That’s it. Happy coding.