Question

Inode limit, how to increase inode limit on block storage volume?

Posted on December 23, 2018
Storage Ubuntu 18.04 Block Storage Linux Commands DigitalOcean
Asked by Matthew Webster

I’ve successfully added a 100gb block storage volume to my droplet, which was originally added to store a few million .json files and a few million image files. The issue I am noticing at the moment is that after the first 3 million .json files the inode used % is already at 85%, even though the storage usage is only at 12%.

seen here: https://imgur.com/a/NEM0e3i

What I am looking to do is reformat the volume and hopefully adjust the inode table to better fit my 100gb disk space and the files I am looking to store. In total I am looking to store around 8-10 million small .json files and around 4-5 million image files.

If possible I am looking for some guidance/how to on how to adjust the inode table to allow storage of these millions of files, ideally from the digitalocean console command line.

Currently I am trying to avoid my default solution of just increasing the volume size to something like 500gb just to increase the inode limit and end up in total using 20% of the 500gb storage and paying 5x the cost.

I am not a server admin in general so most of this is learn as you go/trial and error,

Any help or insight would be appreciated,

Thanks

-Matt

Submit an answer

This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

unixynet • December 26, 2018

Accepted Answer

How small/large are these files? You can set the number of inodes at format-time of the block storage. It can be as high as 4 billion inodes. So partition the new the block storage as ext4 with parted then format it with this additional parameter:

mkfs.ext4 -N 4000000000 /dev/path/to/volume

Then mount it as you mount any volume and rsync the existing json files. I wouldn’t create it with a capacity of 4 billions inodes as that’s a bit extreme. It also depends on the average size of the json files. Maybe set it to 1 or 2 billion max.

Cheers