Handling File Uploads with Hapi.js

Introduction

File uploading is a common feature that almost every website needs. We will go through step by step on how to handle single and multiple file(s) uploads with Hapi, save it to the database (LokiJS), and retrieve the saved file for viewing.

The complete source code is available here: https://github.com/chybie/file-upload-hapi.

We will be using TypeScript throughout this tutorial.

Install Required Dependencies

I am using Yarn for package management. However, you can use npm if you like.

Dependencies

Run this command to install required dependencies

// run this for yarn
yarn add hapi boom lokijs uuid del

// or using npm
npm install hapi boom lokijs uuid del --save

• hapi: We will develop our API using HapiJs
• boom: A plugin for Hapi, HTTP-friendly error objects
• loki: LokiJS, a fast, in-memory document-oriented datastore for node.js, browser and cordova
• uuid: Generate unique id
• del: Delete files and folders

Development Dependencies

Since we are using TypeScript, we need to install typings files in order to have an auto-complete function (IntelliSense) during development.

// run this for yarn
yarn add typescript @types/hapi @types/boom @types/lokijs @types/uuid @types/del --dev

// or using npm
npm install typescript @types/hapi @types/boom @types/lokijs @types/uuid @types/del --save-dev

Setup

A couple of setup steps to go before we start.

TypeScript Configuration

Add a typescript configuration file. To know more about TypSscript configuration, visit https://www.typescriptlang.org/docs/handbook/tsconfig-json.html.

// tsconfig.json

{
    "compilerOptions": {
        "module": "commonjs",
        "moduleResolution": "node",
        "target": "es6",
        "noImplicitAny": false,
        "sourceMap": true,
        "outDir": "dist"
    }
}

1. The compiled JavaScript code will be output to the dist folder.
2. Since Node.js 7.5+ supports ES6 / 2015, we will set the target as es6.

Start Script

Add the following scripts.

// package.json

{
  ...
  "scripts": {
    "prestart": "tsc",
    "start": "node dist/index.js"
  }
  ...
}

Later on, we can run yarn start or npm start to start our application.

1. When we run yarn start, it will trigger prestart script first. The command tsc will read the tsconfig.json file and compile all typescript files to javascript in dist folder.
2. Then, we will run the compiled index file dist/index.js.

Starting Hapi Server

Let’s start creating our Hapi server.

// index.ts

import * as Hapi from 'hapi';
import * as Boom from 'boom';
import * as path from 'path'
import * as fs from 'fs';
import * as Loki from 'lokijs';

// setup
const DB_NAME = 'db.json';
const COLLECTION_NAME = 'images';
const UPLOAD_PATH = 'uploads';
const fileOptions = { dest: `${UPLOAD_PATH}/` };
const db = new Loki(`${UPLOAD_PATH}/${DB_NAME}`, { persistenceMethod: 'fs' });

// create folder for upload if not exist
if (!fs.existsSync(UPLOAD_PATH)) fs.mkdirSync(UPLOAD_PATH);

// app
const app = new Hapi.Server();
app.connection({
    port: 3001, host: 'localhost',
    routes: { cors: true }
});

// start our app
app.start((err) => {

    if (err) {
        throw err;
    }
    console.log(`Server running at: ${app.info.uri}`);
});

The code is pretty expressive itself. We set the connection port to 3001, allow Cross-Origin Resource Sharing (CORS), and start the server.

Upload a Single File

Let’s create our first route. We will create a route to allow users to upload their profile avatar.

Route

// index.ts
...
import {
  loadCollection, uploader
} from './utils';
...

app.route({
  method: 'POST',
  path: '/profile',
  config: {
    payload: {
      output: 'stream',
      allow: 'multipart/form-data' // important
    }
  },
  handler: async function (request, reply) {
    try {
      const data = request.payload;
      const file = data['avatar']; // accept a field call avatar

      // save the file
      const fileDetails = await uploader(file, fileOptions);

      // save data to database
      const col = await loadCollection(COLLECTION_NAME, db);
      const result = col.insert(fileDetails);
      db.saveDatabase();

      // return result
      reply({ id: result.$loki, fileName: result.filename, originalName: result.originalname });

    } catch (err) {
      // error handling
      reply(Boom.badRequest(err.message, err));
    }
  }
});

1. This is an HTTP POST function.
2. We configure the payload to allow multipart/form-data and receive the data as a stream.
3. We will read the field avatar for file upload.
4. We will call uploader function (we will create it soon) to save the input file.
5. Then, we will load the LokiJS images table / collection (we will create loadCollection next) and create a new record.
6. Save the database.
7. Return result.

Load LokiJS Collection

A generic function to retrieve a LokiJS collection if exists, or create a new one if it doesn’t.

// utils.ts

import * as del from 'del';
import * as Loki from 'lokijs';
import * as fs from 'fs';
import * as uuid from 'uuid;

const loadCollection = function (colName, db: Loki): Promise<LokiCollection<any>> {
    return new Promise(resolve => {
        db.loadDatabase({}, () => {
            const _collection = db.getCollection(colName) || db.addCollection(colName);
            resolve(_collection);
        })
    });
}

export { loadCollection }

Uploader Function

Our uploader will handle single file upload and multiple file upload (will create later).

// utils.ts
...

const uploader = function (file: any, options: FileUploaderOption) {
    if (!file) throw new Error('no file(s)');

    return _fileHandler(file, options);
}

const _fileHandler = function (file: any, options: FileUploaderOption) {
    if (!file) throw new Error('no file');

    const orignalname = file.hapi.filename;
    const filename = uuid.v1();
    const path = `${options.dest}${filename}`;
    const fileStream = fs.createWriteStream(path);

    return new Promise((resolve, reject) => {
        file.on('error', function (err) {
            reject(err);
        });

        file.pipe(fileStream);

        file.on('end', function (err) {
            const fileDetails: FileDetails = {
                fieldname: file.hapi.name,
                originalname: file.hapi.filename,
                filename,
                mimetype: file.hapi.headers['content-type'],
                destination: `${options.dest}`,
                path,
                size: fs.statSync(path).size,
            }

            resolve(fileDetails);
        })
    })
}

...

export { loadCollection, uploader }

1. We will read the uploaded file name.
2. We will generate a random UUID as the new file name to avoid name conflicts.
3. We will then stream and write the file to the defined folder. It’s uploads folder for our case.

Run Our Application

You may run the application with yarn start. I try to call the locahost:3001/profile API with (Postman)[https://www.getpostman.com/apps], a GUI application for API testing.

When I upload a file, you can see that a new file is created in uploads folder, and the database file db.json is created as well.

When I issue a call without passing in avatar, an error will be returned.

Filter File Type

We can handle file upload successfully now. Next, we need to limit the file type to image only. To do this, let’s create a filter function that will test the file extensions, then modify our _fileHandler to accept an optional filter option.

// utils.ts

...

const imageFilter = function (fileName: string) {
    // accept image only
    if (!fileName.match(/\.(jpg|jpeg|png|gif)$/)) {
        return false;
    }

    return true;
};

const _fileHandler = function (file: any, options: FileUploaderOption) {
    if (!file) throw new Error('no file');

    // apply filter if exists
    if (options.fileFilter && !options.fileFilter(file.hapi.filename)) {
        throw new Error('type not allowed');
    }

    ...
}

...

export { imageFilter, loadCollection, uploader }

Apply the Image Filter

We need to tell the uploader to apply our image filter function. Add it in fileOptions variable.

// index.ts
import {
    imageFilter, loadCollection, uploader
} from './utils';

..
// setup
...

const fileOptions: FileUploaderOption = { dest: `${UPLOAD_PATH}/`, fileFilter: imageFilter };

...

Restart the application, try to upload a non-image file and you should get an error.

Upload Multiple Files

Let’s proceed to handle multiple files upload now. We will create a new route to allow user to upload their photos.

Route

...

app.route({
    method: 'POST',
    path: '/photos/upload',
    config: {
        payload: {
            output: 'stream',
            allow: 'multipart/form-data'
        }
    },
    handler: async function (request, reply) {
        try {
            const data = request.payload;
            const files = data['photos'];

            const filesDetails = await uploader(files, fileOptions);
            const col = await loadCollection(COLLECTION_NAME, db);
            const result = [].concat(col.insert(filesDetails));

            db.saveDatabase();
            reply(result.map(x => ({ id: x.$loki, fileName: x.filename, originalName: x.originalname })));
        } catch (err) {
            reply(Boom.badRequest(err.message, err));
        }
    }
});

...

The code is similar to single file upload, except we accept a field photos instead of avatar, accept an array of files as input, and reply the result as an array.

Modify Uploader Function

We need to modify our uploader function to handle multiple files upload.

// utils.ts
...

const uploader = function (file: any, options: FileUploaderOption) {
    if (!file) throw new Error('no file(s)');

    // update this line to accept single or multiple files
    return Array.isArray(file) ? _filesHandler(file, options) : _fileHandler(file, options);
}

const _filesHandler = function (files: any[], options: FileUploaderOption) {
    if (!files || !Array.isArray(files)) throw new Error('no files');

    const promises = files.map(x => _fileHandler(x, options));
    return Promise.all(promises);
}

...

Retrieve List of Images

Next, create a route to retrieve all images.

// index.ts
...

app.route({
    method: 'GET',
    path: '/images',
    handler: async function (request, reply) {
        try {
            const col = await loadCollection(COLLECTION_NAME, db)
            reply(col.data);
        } catch (err) {
            reply(Boom.badRequest(err.message, err));
        }
    }
});

...

The code is super easy to understand.

Retrieve Image by Id

Next, create a route to retrieve an image by id.

// index.ts
...

app.route({
    method: 'GET',
    path: '/images/{id}',
    handler: async function (request, reply) {
        try {
            const col = await loadCollection(COLLECTION_NAME, db)
            const result = col.get(request.params['id']);

            if (!result) {
                reply(Boom.notFound());
                return;
            };

            reply(fs.createReadStream(path.join(UPLOAD_PATH, result.filename)))
                .header('Content-Type', result.mimetype); // important
        } catch (err) {
            reply(Boom.badRequest(err.message, err));
        }
    }
});

...

1. We will return 404 if the image does not exist in the database.
2. We will stream the file as output, set the content-type correctly so our client or browser knows how to handle it.

Run the Application

Now restart the application, upload a couple of images, and retrieve it by id. You should see the image is return as an image instead of a JSON object.

Clear All Data When Restart

Sometimes, you might want to clear all the images and database collection during development. Here’s a helper function to do so.

// utils.ts

....

const cleanFolder = function (folderPath) {
    // delete files inside folder but not the folder itself
    del.sync([`${folderPath}/**`, `!${folderPath}`]);
};

...

export { imageFilter, loadCollection, cleanFolder, uploader }

// index.ts

// setup
...

// optional: clean all data before start
cleanFolder(UPLOAD_PATH);
if (!fs.existsSync(UPLOAD_PATH)) fs.mkdirSync(UPLOAD_PATH);

...

Conclusion

Handling file(s) uploads with Hapi is not as hard as you thought.

The complete source code is available here: https://github.com/chybie/file-upload-hapi.

That’s it. Happy coding.