We’ve all been there – needing a program – and what do we do? Most of us just apt-get install postfix and presto! We magically have Postfix installed.
It isn’t really magic, though. The package manager apt-get searches for, downloads, and installs the package for you. This is highly convenient, but what if apt-get can’t find the program you need on its standard list of repositories? Thankfully, apt-get allows users to specify custom download locations (called repositories).
In this tutorial, we will walk through setting up your own secure repository and making it public for others to use. We will be creating the repository on a Ubuntu 14.04 LTS Droplet, and testing the download from another Droplet with the same distribution.
To get the most out of this guide, make sure to check out our tutorial for managing packages with apt-get.
Two Ubuntu 14.04 LTS Droplets
By the end of the guide you will have:
First, we need a valid package signing key. This step is crucial for a secure repository, since we will be digitally signing all the packages. Package signing gives the downloader confidence that the source can be trusted.
In this section, you will generate an encrypted master public key and a signing subkey by following these steps:
Let’s make the master key. This key should be kept safe and secure since this is what people will be trusting.
Before we begin, let’s install rng-tools though apt-get:
apt-get install rng-tools
GPG requires random data, called entropy, to generate keys. Entropy is normally generated over time by the Linux kernel and stored in a pool. However, on cloud servers (like Droplets), the kernel may have trouble generating the amount of entropy required by GPG. To help the kernel, we install the rngd program (found in the rng-tools package). This program will ask the host server (where the Droplets are located) for entropy. Once retrieved, rngd
will add the data to the entropy pool to be used by other applications like GPG.
If you get a message like this:
Trying to create /dev/hwrng device inode...
Starting Hardware RNG entropy gatherer daemon: (failed).
invoke-rc.d: initscript rng-tools, action "start" failed.
Start the rngd daemon manually with:
rngd -r /dev/urandom
By default rngd looks for a special device to retrieve entropy from /dev/hwrng. Some Droplets do not have this device. To compensate we use the pseudo random device /dev/urandom by specifying the -r directive. For more information, you can check out our tutorial: How to Setup Additional Entropy.
Now that we have a pool of entropy, we can generate the master key. Do this by invoking the command gpg. You will see a prompt similar to the following:
gpg --gen-key
gpg (GnuPG) 1.4.16; Copyright (C) 2013 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Please select what kind of key you want:
(1) RSA and RSA (default)
(2) DSA and Elgamal
(3) DSA (sign only)
(4) RSA (sign only)
Your selection? 1
Specify the first option, “RSA and RSA (default)” 1
, in the prompt. Selecting this will have gpg generate first a signing key, then a encryption subkey (both using the RSA algorithm). We don’t need an encryption key for this tutorial, but as a great person once said, “why not?” There is no disadvantage in having both, and you may use the key for encryption in the future.
Hit Enter and you’ll be prompted for a keysize:
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
The key size correlates directly to how secure you want your master key to be. The higher the bit size, the more secure key. The Debian project recommends using 4096 bits for any signing key, so I would specify 4096 here. For the next 2-5 years the default bit size 2048 is sufficient if you’d rather use that. A size of 1024 is uncomfortably close to being unsafe and should not be used.
Press Enter for the expire prompt.
Please specify how long the key should be valid.
0 = key does not expire
<n> = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years
Key is valid for? (0) 0
Master keys do not normally have expiration dates, but set this value as long as you expect to use this key. If you only plan to use this repository for only the next 6 months you can specify 6m. 0 will make it valid forever.
Hit Enter, then y. You will be prompted to generate a “user ID”. This information will be used by others and yourself to identify this key – so use real information!
You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:
"Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>"
Real name: Mark Lopez
Email address: mark.lopez@example.com
Comment:
You selected this USER-ID:
"Mark Lopez <mark.lopez@example.com>"
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
If the information is correct, hit o and Enter. We need to add a password to ensure that only you have access to this key. Make sure to memorize this password since there is no way to recover a gpg key password (a good thing).
You need a Passphrase to protect your secret key.
Enter passphrase: (hidden)
Repeat passphrase: (hidden)
Now for some magic (math) to happen. This might take a little while, so sit back or get a cup of your favorite drink.
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
Not enough random bytes available. Please do some other work to give
the OS a chance to collect more entropy! (Need 300 more bytes)
+++++
................+++++
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
..+++++
+++++
gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: key 10E6133F marked as ultimately trusted
public and secret key created and signed.
gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
pub 4096R/10E6133F 2014-08-16
Key fingerprint = 1CD3 22ED 54B8 694A 0975 7164 6C1D 28A0 10E6 133F
uid Mark Lopez <mark.lopez@example.com>
sub 4096R/7B34E07C 2014-08-16
Now we have a master key. The output shows that we created a master key for signing (`0E6133F on the pub line above). Your key will have different IDs. Make note of your signing key’s ID (the example uses 10E6133F). We’ll need that information in the next steps when creating another subkey for signing.
Now we’ll create a second signing key so that we don’t need the master key on this server. Think of the master key as the root authority that gives authority to subkeys. If a user trusts the master key, trust in a subkey is implied.
In the terminal execute:
gpg --edit-key 10E6133F
Replace the example ID with your key’s ID. This command enters us into the gpg environment. Here we can edit our new key and add a subkey. You’ll see the following output:
gpg (GnuPG) 1.4.16; Copyright (C) 2013 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Secret key is available.
pub 4096R/10E6133F created: 2014-08-16 expires: never usage: SC
trust: ultimate validity: ultimate
sub 4096R/7B34E07C created: 2014-08-16 expires: never usage: E
[ultimate] (1). Mark Lopez <mark.lopez@example.com>
gpg>
At the prompt, type addkey
:
addkey
Press Enter. GPG will prompt for your password. Enter the password that you used to encrypt this key.
Key is protected.
You need a passphrase to unlock the secret key for
user: "Mark Lopez <mark.lopez@example.com>"
4096-bit RSA key, ID 10E6133F, created 2014-08-16
gpg: gpg-agent is not available in this session
Enter passphrase: <hidden>
You will see the following prompt for key type.
Please select what kind of key you want:
(3) DSA (sign only)
(4) RSA (sign only)
(5) Elgamal (encrypt only)
(6) RSA (encrypt only)
Your selection? 4
We want to create a <i>signing</i> subkey, so select "RSA (sign only)” 4
. RSA is faster for the client, while DSA is faster for the server. We’re picking RSA in this case because, for every signature that we make on a package, possibly hundreds of clients will need to verify it. The two types are equally secure.
Again we are prompted for a key size.
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
This tutorial uses 4096 for increased security.
Please specify how long the key should be valid.
0 = key does not expire
<n> = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years
Key is valid for? (0) 1y
We already have a master key, so the expiration time for the subkey is less important. One year is a good time frame.
Hit Enter, and then type y (yes) twice for the next two prompts. Some math will generate another key.
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
............+++++
.+++++
pub 4096R/10E6133F created: 2014-08-16 expires: never usage: SC
trust: ultimate validity: ultimate
sub 4096R/7B34E07C created: 2014-08-16 expires: never usage: E
sub 4096R/A72DB3EF created: 2014-08-16 expires: 2015-08-16 usage: S
[ultimate] (1). Mark Lopez <mark.lopez@example.com>
gpg>
Type save at the prompt.
save
In the output above, the SC from our master key tells us that the key is only for signing and certification. The E means the key may only be used for encryption. Our signing key can be correctly seen with only the S.
Note your new signing key’s ID (the example shows A72DB3EF on the second sub line above). Your key’s ID will be different.
Enter save to return to the terminal and to save your new key.
save
The point of creating the subkey is so we don’t need the master key on our server, which makes it more secure. Now we’ll detach our master key from our subkey. We will need to export the master key and subkey, then delete the keys from GPG’s storage, then re-import just the subkey.
First let’s use the –export-secret-key and –export commands to export the whole key. Remember to use your master key’s ID!
gpg --export-secret-key 10E6133F > private.key
gpg --export 10E6133F >> private.key
By default –export-secret-key and –export will print the key to our console, so instead we pipe the output to a new file (private.key). Make sure to specify your own master key ID, as noted above.
Important: Make a copy of the private.key file somewhere safe (not on the server). Possible locations are on a floppy disk or USB drive. This file contains your private key, your public key, your encryption subkey, and your signing subkey.
After you have backed up this file to a safe location, delete the file:
#back up the private.key file before running this# rm private.key
Now export your public key and your subkey. Make sure to change the IDs to match the master key and the second subkey that you generated (don’t use the first subkey).
gpg --export 10E6133F > public.key
gpg --export-secret-subkeys A72DB3EF > signing.key
Now that we have a backup of our keys we can remove our master key from our server.
gpg --delete-secret-key 10E6133F
Re-import only our signing subkey.
gpg --import public.key signing.key
Check that we no longer have our master key on our server:
gpg --list-secret-keys
sec# 4096R/10E6133F 2014-08-16
uid Mark Lopez <mark.lopez@example.com>
ssb 4096R/7B34E07C 2014-08-16
ssb 4096R/A72DB3EF 2014-08-16
Notice the # after sec. This means our master key is not installed. The server contains only our signing subkey.
Clean up your keys:
rm public.key signing.key
The last thing you need to do is publish your signing key.
gpg --keyserver keyserver.ubuntu.com --send-key 10E6133F
This command publishes your key to a public storehouse of keys – in this case Ubuntu’s own key server. This allows others to download your key and easily verify your packages.
Now let’s get to the point of this tutorial: creating an apt-get repository. Apt-get repositories are not the easiest things to manage. Thankfully R. Bernhard created Reprepro, who used to “produce, manage and sync a local repository of Debian packages” (also known as Mirrorer). Reprepro is under the GNU licence and completely open source.
Reprepro can be installed from the default Ubuntu repositories.
apt-get update
apt-get install reprepro
Configuration for Reprepro is repository-specific, meaning you can have different configurations if you make multiple repositories. Let’s first make a home for our repository.
Make a dedicated folder for this repository and move to it.
mkdir -p /var/repositories/
cd /var/repositories/
Create the configuration directory.
mkdir conf
cd conf/
Create two empty config files (options and distributions).
touch options distributions
Open up the options file in your favorite text editor (nano is installed by default).
nano options
This file contains options for Reprepro and will be read every time Reprepro runs. There are several options that you can specify here. See the manual for the other options.
In your text editor add the following.
ask-passphrase
The ask-passphrase directive tells Reprepro to request a GPG password when signing. If we don’t add this to the options Reprepro will die if our key is encrypted (it is).
Ctrl + x then y and Enter will save our changes and return to the console.
Open the distributions file.
nano distributions
This file has four required directives. Add these to the file.
Codename: trusty
Components: main
Architectures: i386 amd64
SignWith: A72DB3EF
The Codename directive directly relates to the code name of the released Debian distributions and is required. This is the code name for the distribution that will be downloading packages, and doesn’t necessarily have to match the distribution of this server. For example, the Ubuntu 14.04 LTS release is called trusty, Ubuntu 12.04 LTS is called precise, and Debian 7.6 is known as wheezy. This repository is for Ubuntu 14.04 LTS so trusty should be set here.
The Components field is required. This is only a simple repository so set main
here. There are other namespaces such as “non−free” or “contrib” – refer to apt-get for proper naming schemes.
Architectures is another required field. This field lists binary architectures within this repository separated by spaces. This repository will be hosting packages for 32-bit and 64-bit servers, so i386 amd64 is set here. Add or remove architectures as you need them.
To specify how other computers will verify our packages we use the SignWith directive. This is an optional directive, but required for signing. The signing key earlier in this example had the ID A72DB3EF, so that is set here. Change this field to match the subkey’s ID that you generated.
Save and exit from the file with Ctrl + `x then y and Enter.
You have now set up the required structure for Reprepro.
First let’s change our directory to a temporary location.
mkdir -p /tmp/debs
cd /tmp/debs
We need some example packages to work with – wget them with:
wget https://github.com/Silvenga/examples/raw/master/example-helloworld_1.0.0.0_amd64.deb
wget https://github.com/Silvenga/examples/raw/master/example-helloworld_1.0.0.0_i386.deb
These packages were made purely for this guide and contain a simple bash script to prove the functionality of our repository. You can use different packages if you wish.
Running the program ls
should give us this layout:
ls
example-helloworld_1.0.0.0_amd64.deb example-helloworld_1.0.0.0_i386.deb
We now have two example packages. One for 32-bit (i386) computers, another for 64-bit (amd64) computers. You can add them both to our repository with:
reprepro -b /var/repositories includedeb trusty example-helloworld_1.0.0.0_*
The -b argument specifies the “(b)ase” directory for the repository. The includedeb command requires two arguments - < distribution code name > and < file path(s) >
. Reprepro will prompt for our subkey passcode twice.
Exporting indices...
C3D099E3A72DB3EF Mark Lopez <mark.lopez@example.com> needs a passphrase
Please enter passphrase: < hidden >
C3D099E3A72DB3EF Mark Lopez <mark.lopez@example.com> needs a passphrase
Please enter passphrase: < hidden >
Success!
We can list the managed packages with the list command followed by the codename. For example:
reprepro -b /var/repositories/ list trusty
trusty|main|i386: example-helloworld 1.0.0.0
trusty|main|amd64: example-helloworld 1.0.0.0
To delete a package, use the remove command. The remove command requires the codename of the package, and the package name. For example:
reprepro -b /var/repositories/ remove trusty example-helloworld
We now have a local package repository with a couple of packages. Next, we’ll install Nginx as a web server to make this repository public.
Install Nginx
apt-get update
apt-get install nginx
Nginx comes installed with a default example configuration. Make a copy of the file in case you want to look at it at another time.
mv /etc/nginx/sites-available/default /etc/nginx/sites-available/default.bak
touch /etc/nginx/sites-available/default
Now that we have an empty configuration file, we can start configuring our Nginx server to host our new repository.
Open the configuration file with your favorite text editor.
nano /etc/nginx/sites-available/default
And add the following configuration directives:
server {
## Let your repository be the root directory
root /var/repositories;
## Always good to log
access_log /var/log/nginx/repo.access.log;
error_log /var/log/nginx/repo.error.log;
## Prevent access to Reprepro's files
location ~ /(db|conf) {
deny all;
return 404;
}
}
Nginx has some pretty sane defaults. All we needed to configure was the root directory, while denying access to Reprepro’s files. See the in-line comments for more details.
Restart the Nginx service to load these new configurations.
service nginx restart
Your public Ubuntu repository is ready to use!
You’ll need your Droplet’s IP address to let users know the location of the repository. If you don’t know your Droplet’s public address you can find it with ifconfig.
ifconfig eth0
eth0 Link encap:Ethernet HWaddr 04:01:23:f9:0e:01
inet addr:198.199.114.168 Bcast:198.199.114.255 Mask:255.255.255.0
inet6 addr: fe80::601:23ff:fef9:e01/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:16555 errors:0 dropped:0 overruns:0 frame:0
TX packets:16815 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7788170 (7.7 MB) TX bytes:3058446 (3.0 MB)
In the above example, the server’s address is 198.199.114.168. Yours will be different.
With your Reprepro server’s IP address, you can now add this repository to any other appropriate server.
If you haven’t already, spin up another Droplet with Ubuntu 14.04 LTS, so that you can do a test installation from your new repository.
On the new server, download your public key to verify the packages from your repository. Recall that you published your key to keyserver.ubuntu.com.
This is done with the apt-key command.
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 10E6133F
This command downloads the specified key and adds the key to the apt-get database. The adv command tells apt-key to use GPG to download the key. The other two arguments are passed directly to GPG. Since you uploaded your key to “keyserver.ubuntu.com” use the –keyserver keyserver.ubuntu.com directive to retrived the key from the same location. The –recv-keys <key ID> directive specifies the exact key to add.
Now add the repository’s address for apt-get to find. You’ll need your repository server’s IP address from the previous step. This is easily done with the add-apt-repository program.
add-apt-repository "deb http://198.199.114.168/ trusty main"
Note the string that we give add-apt-repository. Most Debian repositories can be added with the following general format:
deb (repository location) (current distribution code name) (the components name)
The repository location should be set to the location of your server. We have an HTTP server so the protocol is http://. The example’s location was 198.199.114.168. Our server’s code name is trusty. This is a simple repository, so we called the component “main”.
After we add the repository, make sure to run an apt-get update. This command will check all the known repositories for updates and changes (including the one you just made).
apt-get update
After updating apt-get, you can now install the example package from your repository. Use the apt-get command normally.
apt-get install example-helloworld
If everything is successful you can now execute example-helloworld and see:
Hello, World!
This package was successfully installed!
Congratulations! You have just installed a package from the repository that you created!
To remove the example package, run this command:
apt-get remove example-helloworld
This removes the example package that you just installed.
In this guide you set out to create a secure APT repository. You’ve learned how to create a secure key to sign packages; how to create and manage a repository with Reprepro; and how to add this repository to another server. Check out the other guides here to further increase your knowledge of the cool things that Open Source and Linux allow you to do.
Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Sign up for Infrastructure as a Newsletter.
Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
Using Ubuntu 16.04 I had to include the "autoindex on; " to the default server config to be able to view the directory listing, otherwise everything was returning 403s:
server {
}
Very thorough. Thank you!
Very nice tutorial. Thanks. So when we create a repo using reprepro command (after setting up the gpg keys), do all the debs in the repo get signed individually or is it the repo as a whole that gets signed?
(my requirement is that I have some 20 odd debian packages & I want to sign all of them & create a private repo during the installation of my product. During installation the packages must be verified for the authenticity.)
Absolutely excellent guide. Thank you so much for being so detailed. It’s very easy to understand. I have a working private repository now!
For those of you trying to add package with
reprepro
via SSH and have su/sudoed to another user, you are likely to expect the following issue:This is due to the fact as
su
does not change the ownership of your TTY, thepinentry
tool used by gpg to decrypt your GPG private key is failing to connect its sdtin to the right TTY and still believes you are the user from which you runsu
/sudo
.To fix this issue, two solutions:
reprepro
command.You should now get the successful following output. As you can see the passphrase is now asked properly.
Ps.: I’m running
reprepro
on Arch Linux that will be my apt server for Debian 9 based hosts (explaining the wordstretch
in the aforementioned example).First of all, great tutorial. Thanks :)
For anyone who tried this tutorial on xenial: Latest releases of reprepro uses GPG v2.1 instead v1.4. On SSH connection I had a long lasting problem importing or creating secret keys using gpg2. For some reason the private-keys folder under .gnupg directory was created with permission 600, therefore gpg2 was unable to write on it. using:
sudo chmod 700 private-keys-v1.d
solved the problem.I tried 3 tutorials, around 4 hours investing in this nonsense and no success. Is there a more simple way to connect to mongo without all those sudo commands? Serious, you have to be expert in ubuntu to make this work. First of all, your console sucks, no copy, and no past, second, all your tutorials do not work, and i followed step by step, repeated few times nothing… is it hard to make it work without all this sudo process? So every time i need a database, i have to go through useless installation tutorials and writing all these code lines in your “sucking” console? I just need a database and not a course how to install it…
Ran into an interesting bug with this configuration yesterday…
In the server block:
I recently added the dbus-python package to my repo, but when I update a client system, it 404’s on dbus-python and dbus-python-dev.
It was driving me crazy, as I knew the files were there, deleted them and readded to make sure nothing had “magically” been corrupted…
Then I looked at the file names and the /etc/ngingx/sites-available/default config, and it jumped out at me - the location rule to prevent access to the “db” directory was matching python-dbus-dev_1.2.0-2build3_all.deb, or any other file that could be expressed as *db*…
I’m going to look at syntax for the nginx conf file, and we should be able to limit the scope so it doesn’t affect these packages, but ONLY the directories in the web root.
Nice guide, but I’m kinda stuck at the moment. I get a bad passphrase error every time I run reprepro, while it doesn’t even ask for a passphrase in the first place. The Packages files aren’t created because of this.
Has anyone here seen something silimar? I’m on Debian jessie
Good tutorial Mark!
I find prefixing --armor when outputting keys is extremely helpful. Then you get ASCII safe versions of the keys instead of the default binary ones. These can be copy pasted and allow inspection of the output (easy tell between public/private keys, etc).
So:
gpg --armor --export
gpg --armor --export-secret-key
etc