How To Use Fabric To Automate Administration Tasks And Deployments

1. Fabric and Python Programming Language
2. System/Server Administration
3. Application Deployment

1. run (fabric.operations.run)
2. sudo (fabric.operations.sudo)
3. local (fabric.operations.local)
4. get (fabric.operations.get)
5. put (fabric.operations.put)
6. prompt (fabric.operations.prompt)
7. reboot (fabric.operations.reboot)

1. cd (fabric.context_managers.cd)
2. lcd (fabric.context_managers.lcd)
3. path (fabric.context_managers.path)
4. settings (fabric.context_managers.settings)
5. prefix (fabric.context_managers.prefix)

As we have briefly mentioned above, although Fabric can be used in a very large scale of scenarios, it is a Python based library and fabfiles need to be programmed using the Python Programming Language.

Regardless of your experience with any other programming language (including Python), as you make your way through our Fabric articles, you will learn how to work with this tool and it shall not take you long to see how simplistic and wonderful it is.

Python is an extremely popular, widely adopted general purpose (i.e. not created to solve a specific problem) programming language. It can be easily distinguished by the importance it puts on code readability and simplicity. To understand Python, check out the very short Python Enhancement Proposal (PEPs) 20 The Zen of Python, followed by bit long Style Guide for Python Code.

In order to have an overall understanding of what programming in Python might be like, you can read a few of the great articles listed at Python Beginner’s Guide.

One of the key areas for using Fabric is automating the everyday tasks of system (and server) administration. These jobs include pretty much everything that relates to:

• Building a server;

• Its maintenance, and;

• Monitoring.

When you begin working with your very own droplet (which is a fully-fledged virtualised server with full control / access), things that appear as a mystery will quickly start to become familiar to you. As you deploy your applications and start dealing with their maintenance, it is only natural to expect that you will be running into some issues. However, when your application gains popularity and things start to grow, the need of managing multiple droplets and repeating everything over and over again ceases to become fun.

That is exactly when you will wish you had met Fabric years ago.

Deploying an application (regardless of it being a web site, an API, or a server) usually means setting up a system from scratch (or from a snapshot taken in time), preparing it by updating everything, downloading dependencies, setting up the file structure and permissions, followed by finally uploading your codebase - or downloading it using a SCM such as Git.

During the development process, you are also likely to have commands that need to be routinely executed (ex: right before entering a deployment cycle).

Being able to script these tasks (both local and remote) in a logically organized and – most importantly – programmable manner proves to be invaluable shortly after you realize how much time is being wasted repeating the same steps constantly, rendering everything error-prone during the process.

This is exactly when Fabric comes to your aid in the form of a Python file that will know what to do and where to do it.

Fabric’s run procedure is used for executing a shell command on one or more remote hosts.

• The output results of run can be captured using a variable.

• If command succeeded or failed can be checked using .failed and .succeeded.

Usage examples:

# Create a directory (i.e. folder)
run("mkdir /tmp/trunk/")

# Uptime
run("uptime")

# Hostname
run("hostname")

# Capture the output of "ls" command
result = run("ls -l /var/www")

# Check if command failed
result.failed

Along with run, the most widely used Fabric command is probably sudo. It allows the execution of a given set of commands and arguments with sudo (i.e. superuser) privileges on the remote host.

If sudo command is used with an explicitly specified user, the execution will happen not as root but another (i.e. UID 1010).

Usage examples:

# Create a directory
sudo("mkdir /var/www")

# Create a directory as another user
sudo("mkdir /var/www/web-app-one", user="web-admin")

# Return the output
result = sudo("ls -l /var/www")

As we have mentioned in our introduction, a single Fabric script (fabfile) can be used to perform actions both on the local machine and remote system(s). For this purpose, Fabric provides the local operative to run commands locally.

Unlike run or sudo, however, interacting with the output of local the same way is not possible. Either output can be captured or printed – the switch can be set with capture argument.

Local helpers such as the lcd context manager (which is used for setting the local working directory) are honoured with local, the same way run (or sudo) honours the cd context manager.

Usage examples:

# Create a source distribution tar archive (for a Python App.)
local("python setup.py sdist --formats=gztar", capture=False)

# Extract the contents of a tar archive
local("tar xzvf /tmp/trunk/app.tar.gz")

# Remove a file
local("rm /tmp/trunk/app.tar.gz")

The get command exists to download (i.e. pull) file(s) from the remote system to the computer where the Fabric is being used. It is similar to how scp works and comes in handy when you need to download backups, logging data or some other server related items.

• You can specify the remote path with the remote_path argument.

• You can specify the local - download - path with the local_path argument.

Usage examples:

# Download some logs
get(remote_path="/tmp/log_extracts.tar.gz", local_path="/logs/new_log.tar.gz")

# Download a database back-up
get("/backup/db.gz", "./db.gz")

When you need to upload files, put command can be used very similarly to get. You can again access the results of command’s execution with .failed or .succeeded.

• local_path - set the local path.

• remote_path - set the remote path.

• use_sudo - upload the file to anywhere on the remote machine using a nifty trick: upload to a temporary location then move.

• mode - set the file mode (flags).

• mirror_local - set the file flags (i.e. make executable) automatically by reading the local file’s mode.

Usage examples:

# Upload a tar archive of an application
put("/local/path/to/app.tar.gz", "/tmp/trunk/app.tar.gz")

# Use the context manager `cd` instead of "remote_path" arg.
# This will upload app.tar.gz to /tmp/trunk/
with cd("/tmp"):
    put("local/path/to/app.tar.gz", "trunk")

# Upload a file and set the exact mode desired
upload = put("requirements.txt", "requirements.txt", mode=664)

# Verify the upload
upload.succeeded

When you find yourself in need of some extra flexibility working with Fabric, prompt will come to your rescue. This command does exactly what its name suggests and asks the user (i.e. one that is running the script) to input a certain data to use during the successive execution.

If you are using a single file to manage with multiple applications, for example, you can use prompt to set one to perform the actions.

Before starting with anything, prompt can also be used to query the port number to use.

Usage examples:

# Prompt the user
port_number = prompt("Which port would you like to use?")

# Prompt the user with defaults and validation
port_number = prompt("Which port?", default=42, validate=int)

The reboot command is also self explanatory: it is used to reboot the remote system. By default, it waits two minutes (i.e. 120 seconds -> wait=120) before doing its job.

Usage examples:

# Reboot the remote system
reboot()

# Reboot after 30 seconds
reboot(wait=30)

cd context manager allows keeping the directory state (i.e. where the following block of comments are to be executed). It is similar to running the cd command during an SSH session and running various different commands.

Usage examples:

# The *cd* context manager makes enwrapped command's
# execution relative to the stated path (i.e. "/tmp/trunk")
with cd("/tmp/trunk"):
    items = sudo("ls -l")

# It is possible to "chain" context managers
# The run commands gets executed, therefore at "/tmp/trunk"
with cd("/tmp"):
    with cd("/trunk"):
        run("ls")

The lcd context manager (local cd) works very similarly to one above (cd); however, it only affects the local system’s state.

Usage examples:

# Change the local working directory to project's
# and upload a tar archive
with lcd("~/projects/my_project"):
    print "Uploading the project archive"
    put("app.tar.gz", "/tmp/trunk/app.tar.gz")

path context managers alters the PATH variable.

When you need to temporarily (i.e. for a certain command chain), you can use the settings statement (i.e. override env values).

Usage examples:

# Perform actions using a different *user*
with settings(user="user1"):
    sudo("cmd")

prefix statement does what its name suggests and wraps given run and sudo command with the specified one.

Usage examples:

with prefix("cmd arg."):
    run("./start")
# cmd arg. && ./start