How To Create a calibre Ebook Server on Ubuntu 14.04

Introduction

Calibre is a free and open source ebook manager.

Although Calibre is probably better known for its desktop client, it can also act as a powerful server, allowing you to access your ebooks from anywhere in the world (or share your collection with friends). Keeping your ebooks on a server is great, as you aren’t reliant on having the same reading device with you whenever you want to read. And if you go traveling, you don’t need to worry about taking your ebook collection with you!

The server includes a simple and elegant browser front-end that allows you to search for and download books from your library. It also has a mobile-friendly site built in, making it easy to download books straight to an e-reader – even to ones with only the most basic web functionality.

For example, Calibre’s browser works with the Kindle Touch, which can download books directly even though the device only has an e-ink display and an experimental browser.

In this tutorial we’ll look at how to install, set up, and use Calibre on a Ubuntu 14.04 server. We’ll also take a look at how to use the calibredb command to create, customize, and maintain your ebook database right from the server.

For this tutorial we’ll cover:

• Installing Calibre
• Creating an ebook library, or importing an existing one
• Making Calibre server a background service
• Automatically adding new books to the library

By the end of this tutorial, you’ll have a small initial library to which you can easily add new books!

Prerequisites

Please make sure you have these prerequisites:

• Ubuntu 14.04 Droplet
• A sudo user

Examples in this tutorial are shown for a Droplet running a fresh installation of Ubuntu 14.04, but they should be easily adaptable to other operating systems.

Step 1 — Installing Calibre

Calibre is available from the APT software repositories, but as advised by its creators it is far better to install from the binaries provided on their website. Calibre is updated very frequently and the version in the repos tends to lag behind.

Luckily, the creators of Calibre have made this very simple to do. Just run the following Python command on your server. Before running the command, please double-check the official Calibre site in case the command has been changed.

Install Calibre (make sure you scroll to get the entire command):

sudo -v && wget -nv -O- https://raw.githubusercontent.com/kovidgoyal/calibre/master/setup/linux-installer.py | sudo python -c "import sys; main=lambda:sys.stderr.write('Download failed\n'); exec(sys.stdin.read()); main()"

You will notice some warnings about failed desktop integration, but these are safe to ignore, since you are installing Calibre on a remote server.

Step 2 — Installing Dependencies

The Calibre command line tool calibredb is used for various operations on your Calibre library, such as adding or importing books, and fetching metadata and covers for books.

We’ll take a look at how to use some of these commands later, but for now we’ll just install two dependencies. The first is ImageMagick, without which calibredb won’t run; and the second is xvfb which we’ll use to run calibredb in a virtual X display server – in order to sidestep issues caused by running Calibre in a non-display environment.

To install these just run the following commands.

Update your package lists:

sudo apt-get update

Install xvfb:

sudo apt-get install xvfb

Install ImageMagick:

sudo apt-get install imagemagick

Step 3 — Creating the Library

Now we’re almost ready to start running the server. We need to get some books to serve, however.

You may well have your own ebook library already, so we’ll look at two ways of doing this.

1. Add ebook files directly; we’ll grab a couple from Project Gutenberg
2. Import an existing Calibre library; useful if you’re already running the desktop version of Calibre

Getting Books

First let’s make a directory for our Calibre library. This example creates the directory in your user’s home directory, although you could place it anywhere on the server. Run the following commands:

mkdir ~/calibre-library
mkdir ~/calibre-library/toadd

We’ve created two directories: the first, ~/calibre-library is the one that Calibre will organize automatically, while we’ll add books manually to the toadd sub-directory. Later, we’ll take a look at how to automate this process too.

How we’ll grab some books from Project Gutenberg. For this tutorial we’ll download Pride and Prejudice by Jane Austen and A Christmas Carol by Charles Dickens.

Change to the toadd directory to get started.

cd ~/calibre-library/toadd

Download the two ebooks:

wget http://www.gutenberg.org/ebooks/1342.kindle.noimages -O pride.mobi
wget http://www.gutenberg.org/ebooks/46.kindle.noimages -O christmascarol.mobi

Calibre relies somewhat on file extensions to correctly add books, so the -O flag in the wget command specifies a more friendly filename. If you downloaded a different format from Gutenberg (such as .epub) then you need to change the file extension accordingly.

Adding the Books to Calibre’s Database

Now we need to add these books to the Calibre database using the calibredb command through the xvfb virtual display we installed earlier. To do this, run:

xvfb-run calibredb add ~/calibre-library/toadd/* --library-path ~/calibre-library

The asterisk means that Calibre will add all books found in the toadd directory to the library, in the calibre-library directory. You might see an error about not finding a cover (we chose to download the .mobi files without images), but you should also see confirmation that the books were added to the Calibre database.

Sample output:

Failed to read MOBI cover
Backing up metadata
Added book ids: 1, 2
Notifying calibre of the change

That’s all we need to start seeing the first results. Let’s test out the server. Run:

calibre-server --with-library ~/calibre-library

The command won’t produce any output, but will just appear to hang in your terminal. This is fine for now; we’ll look at daemonizing it properly later. Now open a web browser and navigate to:

• http://your_server_ip:8080

Replace your_server_ip with your Droplet’s IP address. You should see the main page of your library, looking similar to the screenshot below.

If you click on the All books link, you should see the two books that we added earlier. You can click on the Get button below either book to download it.

Uploading an Existing Calibre Library

If you’re already running the desktop version of Calibre and already have your library set up, you can import it to your server easily.

Double-check your current library folder for a file called metadata.db. If this file exists, then everything should just work without any additional configuration.

Upload your entire library folder to your server.

Then, run this command:

calibre-server --with-library /path/to/calibre-library

This will add your existing library in its entirety to the server. You can add more books to it on the server by placing the book files in the toadd directory, as explained in this tutorial.

Step 4 — Making Calibre a Background Service

We don’t really want to keep a shell open with the calibre-server command running in it just to keep the server running.

While we could add the --daemonize flag to the command, there are better ways to do it. Below we’ll look at how easy it is to make calibre-server into a service so that it will automatically start on system reboot and so that we can very easily start, stop, or restart the process.

Until recently, the way to achieve this was to write complex scripts and put them in the /etc/init.d/ directory. The currently recommended way is to use a far simpler Upstart script, which is a .conf file placed in the /etc/init/ directory. We’ll take a look at how to do this:

If the server is still running, hit CTRL + C in your terminal to stop it.

Now create a new configuration file:

sudo nano /etc/init/calibre-server.conf

Create the Upstart script, being sure to replace the variables marked in red:

description "Calibre (ebook manager) content server"

start on runlevel [2345]
stop on runlevel [^2345]

respawn

env USER='myusername'
env PASSWORD='mypassword'
env LIBRARY_PATH='/home/user/calibre-library'
env MAX_COVER='300x400'
env PORT='80'

script
    exec /usr/bin/calibre-server --with-library $LIBRARY_PATH --auto-reload \
                                 --max-cover $MAX_COVER --port $PORT \
                                 --username $USER --password $PASSWORD
end script

Paste this into your text editor and save it. (CTRL + X, then Y, then ENTER). We’ll look at what each line does below:

• The first line is just a description to help you (or others) know what the script does
• The next two lines state at what level you want your script to start and stop on, since Upstart, allows for order specification so that scripts that rely on each other will start in the right order. Level 1 is for all essential services, so we’ll start on level 2, by which time we know that the network and anything else we need will be up and running
• respawn means that if the service stop unexpectedly, it’ll try to restart

The next lines are all variables that we pass to the calibre-server command. Before, we just used the minimum of specifying the --with-library option, but we can see now how much flexibility Calibre offers. Above, we’ve specified:

• Username and password to access the library from the web (please change these from the examples provided)
• Library location path, as before
• Max image size for book cover images (this is useful to make the page load more quickly)
• Port number (here we’ve changed it to 80; change this to something else if you already use port 80 to serve standard web pages, etc.)
• Finally, in the script section (known as a stanza) we run the main command using exec, and passing in all our variables. The /usr/bin/calibre-server part is the path to the executable

Once you’ve saved the script and closed the editor, start up the server:

sudo start calibre-server

This time you should see this output, but with a different process number:

calibre-server start/running, process 7811

Now use a browser to navigate to your server’s IP address or domain name.

You should see a popup form asking for the username and password. These should be the ones you added to the Upstart script. Enter these and you’ll be taken to your ebook library as before.

The server can now easily be stopped, started, and restarted using the following commands:

sudo service calibre-server stop
sudo service calibre-server start
sudo service calibre-server restart

This makes managing the server a lot easier than having to manually deal with daemon processes and process IDs!

The site by default has a mobile version that works nicely with smaller-screen devices such as phones and e-readers. This should load automatically if you visit the site from a mobile device.

Step 5 — Creating a Cron Job to Add Books Automatically

We can write a simple cron job to watch our toadd directory for new books.

Every 10 minutes it will look for files in the /home/user/calibre-library/toadd/ directory, add any files in there to our Calibre database, and then remove the original files. (Calibre makes copies of the files when it adds them to our library so we don’t need the originals once the add has taken effect.) This means that if you transfer book files via scp, ssh, etc. to this directory from your main machine, or just download them directly into the toadd directory, then they’ll automatically be added to your Calibre database and be available for download from your library!

To create a cron job, execute:

crontab -e

You might have to make a selection about your preferred text editor.

At the end of the file add the line:

*/10 * * * * xvfb-run calibredb add /home/user/calibre-library/toadd/ -r --with-library /home/user/calibre-library && rm /home/user/calibre-server/toadd/*

The first part of the command (*/10 * * * *) means that the command should be run every ten minutes. The second part is the same as the command we manually ran earlier. It adds all the books from the toadd folder to the database and then removes the original files.

That’s that. You can now access your ebooks from anywhere in the world.

Note: The search results in Calibre aren’t sorted by relevance, so if you enter a common term you often find unrelated books before the one you’re looking for. However, you can specify to search only by title or author, which does help a lot, and the browse options (browse alphabetically by Author, for example) are very well implemented as well.

Conclusion

There are one or two things to keep in mind about running and maintaining a Calibre server. We’ll take a brief look at these to finish off.

Copyright

If you are only hosting books from Gutenberg or similar sites (i.e., books that are out of copyright), then there is little to say. Just make sure you follow the Gutenberg terms of service. Specifically, if you give access to your book collection to others, make sure to read the section of Gutenberg’s TOS regarding redistribution.

If you are hosting commercially-purchased books, remember that they’ll probably have DRM (Digital Rights Management) and will therefore only be readable from your registered device.

It goes without saying that you should never host pirated or illegal books on your ebook server.

Updates

Calibre pushes updates very frequently. Although most of these are bug fixes and functionality updates, some might be to do with security. It is therefore recommended that you keep up with the updates.

If an important update is published you should manually update the server software. (Again, the APT repos tend to lag behind, so it is not recommended to rely on them for updates).

Security

Even if you don’t publish the IP address of your Droplet, it may be discovered by scripts scanning for open ports. Because the Calibre login functionality does not allow automatic lockout after a number of incorrect attempts, there is a possibility of a brute force attack. To mitigate against this, it is highly recommended that you:

• Don’t use a common username such as admin, calibre, or ebooks
• Don’t use a common or short password, and definitely don’t use your username as a password
• Consider running Calibre on a non-standard port, instead of port 80

And that brings our tutorial to a close. We hope that you enjoy accessing your ebooks from any location or device!

sudo apt-get install ImageMagick 

sudo apt-get install imagemagick

No write acces to /home/qerim/.config/calibre using a temporary dir instead
Traceback (most recent call last):
  File "site.py", line 66, in main
  File "site-packages/calibre/library/cli.py", line 1604, in main
  File "site-packages/calibre/library/cli.py", line 437, in command_add
  File "site-packages/calibre/library/cli.py", line 68, in get_db
  File "site-packages/calibre/db/legacy.py", line 73, in __init__
  File "site-packages/calibre/db/legacy.py", line 46, in create_backend
  File "site-packages/calibre/db/backend.py", line 362, in __init__
  File "site-packages/calibre/db/schema_upgrades.py", line 18, in __init__
  File "site-packages/calibre/db/backend.py", line 807, in execute
  File "src/cursor.c", line 231, in resetcursor
ReadOnlyError: ReadOnlyError: attempt to write a readonly database

setuid *username*
setgid *username*