How to Deploy Pyramid Based Python WSGI Web-Applications

Introduction

---

When you first start working with developing a Python web-application, especially if you are coming from a different language or have a background in desktop (GUI) applications, you might be a little lost in the process of getting your new creation online.

In this DigitalOcean article, following our previous one on getting started with Pyramid and deploying Python web applications using different web servers, we will see about packaging a Pyramid WSGI web application, creating a list of its dependencies, and deploying it (i.e. publishing it online).

Note: This article’s examples, albeit giving a usage example as well, follow the naming conventions used in the getting started article. If you have questions about beginning web application development using Pyramid, or simply want to get familiar with the names used previously, you can check out the article here.

Glossary

---

<h3>1. Pyramid in Brief</h3><hr>

1. Web Application Deployment
2. WSGI
3. On Using a Nginx as a Reverse Proxy

<h3>2. Deploying Pyramid Based WSGI Web-Applications</h3><hr>

1. Updating The System
2. Setting up Python, pip and virtualenv
3. Preparing A Simple Application With The “app” Object Exposed
4. Exposing A Scaffolded Application Object Using .ini Settings
5. Handling Application Dependencies Using pip
6. Creating A List of Application Dependencies
7. Downloading From A List of Application Dependencies

<h3>3. Setting Up Python WSGI Web Application Servers</h3><hr>

1. Serving Pyramid Applications Using CherryPy Web-Server
2. Running and Managing The Application Server

<h3>4. Setting Up Nginx</h3><hr>

1. Installing Nginx
2. Configuring Nginx

Pyramid in Brief

---

Compared to some of its heavier counterparts, Pyramid is one of the lightweight Python web-application frameworks. However, unlike Flask or other “micro” ones, Pyramid still comes with a lot of features and functionality out of the box.

Being an excellent project that is very well maintained, Pyramid has quite a bit of popularity and there are several different ways (web servers and methods) to deploy web-applications based on it.

Web Application Deployment

---

In regards to all Python WSGI web applications, deployment consists of preparing a WSGI module that contains a reference to your application object which is then used as a point of entrance by the web-server to pass the requests.

WSGI

---

WSGI, in a nutshell, is an interface between a web server and the application itself. It exists to ensure a standardized way between various servers and applications (frameworks) to work with each other, allowing interchangeability when necessary (e.g. switching from development to production environment), which is a must-have need nowadays.

Note: If you are interested in learning more about WSGI and Python web servers, check out our article: A Comparison of Web Servers for Python Based Web Applications.

On Using a Nginx as a Reverse Proxy

---

Nginx is a very high performant web server / (reverse)-proxy. It has reached its popularity due to being light weight, relatively easy to work with, and easy to extend (with add-ons / plug-ins). Thanks to its architecture, it is capable of handling a lot of requests (virtually unlimited), which - depending on your application or website load - could be really hard to tackle using some other, older alternatives.

Remember: “Handling” connections technically means not dropping them and being able to serve them with something. You still need your application and database functioning well in order to have Nginx serve client’s responses that are not error messages.

Python WSGI Web Application servers

---

Python web application servers are [usually] either stand-alone C-based solutions or fully (or partially) Python based (i.e. pure-Python) ones.

They operate by accepting a Python module containing - as previously explained - an application callable to contain the web-application and serve it on a network.

Although some of them are highly capable servers that can be used directly, it is recommended to use Nginx in front for the reasons mentioned above (e.g. higher performance). Similarly, development servers that are usually shipped with web application frameworks are not recommended to be used in production due to their lack of functionality - with a few exceptions, of course!

Some Popular Python WSGI web servers are:

• CherryPy

• Gunicorn

• uWSGI

• waitress

Deploying Pyramid Based WSGI Web-Applications

---

Thanks to Python’s WSGI middleware specification, for all applications that are set to run on the protocol, there are many choices in terms of web application servers.

Recap: Updating The System

---

In order to have a stable deployment server, it is crucial to keep things up-to-date and well maintained.

To ensure that we have the latest available versions of default applications, we need to update our system.

For Debian Based Systems (i.e. Ubuntu, Debian), run the following:

aptitude    update
aptitude -y upgrade

For RHEL Based Systems (i.e. CentOS), run the following:

yum -y update

Tip: With CentOS, upgrade means upgrading the operating system. update, however, updates the applications. With Debian/Ubuntu, update updates the list of application sources and upgrade upgrades them to their newer versions.

Recap: Setting up Python, pip and virtualenv

---

Note for CentOS / RHEL Users:

CentOS / RHEL, by default, comes as a very lean server. Its toolset, which is likely to be dated for your needs, is not there to run your applications but to power the server’s system tools (e.g. YUM).

In order to prepare your CentOS system, Python needs to be set up (i.e. compiled from the source) and pip / virtualenv need installing using that interpreter.

To learn about How to Set Up Python 2.7.6 and 3.3.3 on CentOS 6.4 and 5.8, with pip and virtualenv, please refer to: How to Set Up Python 2.7.6 and 3.3.3 on CentOS.

On Ubuntu and Debian, a recent version of Python interpreter which you can use comes by default. It leaves us with only a limited number of additional packages to install:

• python-dev – development tools

• pip – to manage packages

• virtualenv – to create isolated, virtual environments

python-dev:

---

python-dev is an operating-system level package which contains extended development tools for building Python modules.

Run the following command to install python-dev using aptitude:

aptitude install python-dev

pip:

---

pip is a package manager which will help us to install the application packages that we need.

Run the following commands to install pip:

curl https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py | python -
curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python -
export PATH="/usr/local/bin:$PATH"

You might need sudo privileges.

virtualenv:

---

It is best to contain a Python application within its own environment together with all of its dependencies. An environment can be best described (in simple terms) as an isolated location (a directory) where everything resides. For this purpose, a tool called virtualenv is used.

Run the following to install virtualenv using pip:

sudo pip install virtualenv

Preparing A Simple Application With The “app” Object Exposed

---

Note: This section focuses on getting our sample single-page application online from the previous tutorial.

Currently, we have a single page application (i.e. application.py) that is built to serve “Hello world!” on port 8080 using Pyramid’s development server.

Our current working directory is located at ~/pyramid_sites/hello_world

Our current Pyramid application example (application.py) looks like:

from wsgiref.simple_server import make_server
from pyramid.config import Configurator
from pyramid.response import Response

def hello_world(request):
    return Response('<h1>Hello world!</h1>')

if __name__ == '__main__':
    config = Configurator()
    config.add_view(hello_world)
    app = config.make_wsgi_app()
    server = make_server('0.0.0.0', 8080, app)
    server.serve_forever()

In order to turn this example to a module containing a WSGI callable, we can make the following modifications:

from wsgiref.simple_server import make_server
from pyramid.config import Configurator
from pyramid.response import Response

def hello_world(request):
    return Response('<h1>Hello world!</h1>')

# Move the application object here.
# Create a configurator to make *wsgi app(lication)*
config = Configurator()
config.add_view(hello_world)

# The "app" object to be exposed
app = config.make_wsgi_app()

# If run directly, still construct a [development]
# server process and start serving on port #8080. 
if __name__ == '__main__':
    server = make_server('0.0.0.0', 8080, app)
    server.serve_forever()

After the amendments, press CTRL+X and confirm with Y to save and exit.

From now on, we can pass this file as a module with the application callable to any Python WSGI web server.

Exposing A Scaffolded Application Object Using .ini Settings

---

Alternatively, if you have created a more complex application using Pyramid’s scaffolding, you can build a wsgi.py to contain an application that uses Pyramid’s “.ini” files to load the deployment (or development) configurations.

Note: This file needs to reside inside your root application folder, alongside requirements.txt or your .ini files. It works by including your application as an object.

Note: You can technically choose any name (“including application.py”) for this file. However, wsgi.py is probably - and conventionally - more acceptable than a different one.

To create the wsgi.py file using the nano text editor, run the following:

nano wsgi.py

Copy and paste the below contents:

from pyramid.config import Configurator
from pyramid.response import Response
from pyramid.paster import get_app

def hello(request):
    return Response('Hello!')

config = Configurator()
config.add_route('hello', '/')
config.add_view(home, route_name='hello')

app = config.make_wsgi_app()

# Or from an .ini file:
# app = get_app('config.ini', 'application_name')

Handling Application Dependencies Using pip

---

Since it is highly likely that you have started the development process on a local machine, when deploying your application, you will need to make sure that all of its dependencies are installed (inside your virtual environment).

Creating A List of Application Dependencies

---

The simplest way to get the dependencies on the production environment is by using pip. With a single command, it is capable of generating all the packages (or dependencies) you have installed (within your activated environment, if not, globally on your system) and again with a single command, it allows you to have them all downloaded and installed.

Note: This section contains information which is to be executed on your local development machine or from wherever you want to generate the list of application dependencies. This file should be placed inside your application directory and uploaded to your server.

Using “pip” to create a list of installed packages:

pip freeze > requirements.txt

This command will create a file called “requirements.txt” which contains the list of all installed packages. If you run it within a virtualenv, the list will consist of packages installed inside the environment only. Otherwise, all packages, installed globally will be listed.

Downloading From A List of Application Dependencies

---

Using pip to install packages from a list:

Note: This section contains information which is to be executed on your production (i.e. deployment) machine / environment.

pip install -r requirements.txt

This command will download and install all the listed packages. If you are working within an activated environment, the files will be downloaded there. Otherwise, they will be installed globally - which is not the recommended way for the reasons explained in the previous sections.

Setting Up Python WSGI Web Application Servers

---

After setting up our Pyramid application to expose its app object, we can begin the actual deployment process with downloading and installing our web application server of choice.

In this article, we will focus on using CherryPy due to its capabilities and combined simplicity.

Note: Instructions given here are brief. To learn more, check out our how-to article on pip and virtualenv Common Python Tools: virtualenv and pip. If you are working with a CentOS based server, you might want to see How to Set Up Python 2.7.6 and 3.3.3 on CentOS article as well.

Serving Pyramid Applications Using CherryPy Web-Server

---

CherryPy’s pure Python web server is a compact solution which comes with the namesake framework. Defined by the project as a “high-speed, production ready, thread pooled, generic HTTP server,” it is a modularized component which can be used to serve any Python WSGI web application.

Following our first Pyramid tutorial, we can continue to work inside the virtual environment (~/pyramid_sites/env) and install CherryPy there. If you have created a different one at a different location, you can do it there as well.

# Install CherryPy Framework and HTTP Web-Server
pip install cherrypy

Upon installing the application package, in order to serve your Pyramid application, you need to create a “server.py” file for your Phython interpreter to run. It is this file that includes your application package and serves it using CherryPy’s web server.

To create a server.py file, run the following:

nano server.py

Copy and paste the below contents to create a sample server application using the CherryPy’s HTTP Server:

# Import your application as:
# from wsgi import application
# Example:

# If you are using the wsgi.py (standard Pyramid)
# from wsgi import app

# If using application.py (single page example):
from application import app

# Import CherryPy
import cherrypy

if __name__ == '__main__':

    # Mount the application (or *app*)
    cherrypy.tree.graft(app, "/") 
                
    # Unsubscribe the default server
    cherrypy.server.unsubscribe()

    # Instantiate a new server object
    server = cherrypy._cpserver.Server()
    
    # Configure the server object
    server.socket_host = "0.0.0.0"
    server.socket_port = 8080
    server.thread_pool = 30
    
    # For SSL Support
    # server.ssl_module            = 'pyopenssl'
    # server.ssl_certificate       = 'ssl/certificate.crt'
    # server.ssl_private_key       = 'ssl/private.key'
    # server.ssl_certificate_chain = 'ssl/bundle.crt'
    
    # Subscribe this server
    server.subscribe()
    
    # Example for a 2nd server (same steps as above):
    # Remember to use a different port
    
    # server2             = cherrypy._cpserver.Server()
    
    # server2.socket_host = "0.0.0.0"
    # server2.socket_port = 8080
    # server2.thread_pool = 30
    # server2.subscribe()
    
    # Start the server engine (Option 1 *and* 2)
    
    cherrypy.engine.start()
    cherrypy.engine.block()

Save and exit again by pressing CTRL+X and confirming with Y.

Running and Managing The Application Server

---

To start serving your application, you just need to execute server.py using your Python installation.

Run the following to start the server as configured:

python server.py

This will run the server on the foreground. If you would like to stop it, press CTRL+C.

To run the server in the background, use the following:

python server.py &

When you run an application in the background, you will need to use a process manager (e.g. htop) to kill (or stop) it.

Setting Up Nginx

---

Installing Nginx

---

Note for CentOS / RHEL Users:

The below instructions will not work on CentOS systems. Please see the instructions here for CentOS.

Run the following command to install Nginx using aptitude:

sudo aptitude install nginx

To run Nginx, use the following:

sudo service nginx start

To stop Nginx, use the following:

sudo service nginx stop

To restart Nginx, use the following:

# After each time you reconfigure Nginx, a restart
# or reload is needed for the new settings to come
# into effect.  
sudo service nginx restart

Note: To learn more about Nginx on Ubuntu, please refer to our article: How to Install Nginx on Ubuntu 12.04.

Configuring Nginx

---

Note: Below is a short tutorial on using Nginx as a reverse proxy. To learn more about Nginx, check out How to Configure Nginx Web Server on a VPS.

After choosing and setting up a web server to run our application, we can continue with doing the same with Nginx and prepare it to talk with the back-end server(s) [running the WSGI app].

To achieve this, we need to modify Nginx’s configuration file: nginx.conf

Run the following command to open up nginx.conf and edit it using nano text editor:

sudo nano /etc/nginx/nginx.conf

You can replace the file with the following example configuration to get Nginx work as a reverse-proxy, talking to your application.

Copy and paste the below example configuration:

worker_processes 1;

events {

    worker_connections 1024;

}

http {

    sendfile on;
    
    gzip              on;
    gzip_http_version 1.0;
    gzip_proxied      any;
    gzip_min_length   500;
    gzip_disable      "MSIE [1-6]\.";
    gzip_types        text/plain text/xml text/css
                      text/comma-separated-values
                      text/javascript
                      application/x-javascript
                      application/atom+xml;

    # Configuration containing list of application servers
    upstream app_servers {
    
        server 127.0.0.1:8080;
        # server 127.0.0.1:8081;
        # ..
        # .
    
    }

    # Configuration for Nginx
    server {
    
        # Running port
        listen 80;

        # Settings to serve static files 
        location ^~ /static/  {
        
            # Example:
            # root /full/path/to/application/static/file/dir;
            root /app/static/;
        
        }
        
        # Serve a static file (ex. favico)
        # outside /static directory
        location = /favico.ico  {
        
            root /app/favico.ico;
        
        }

        # Proxy connections to the application servers
        # app_servers
        location / {
        
            proxy_pass         http://app_servers;
            proxy_redirect     off;
            proxy_set_header   Host $host;
            proxy_set_header   X-Real-IP $remote_addr;
            proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header   X-Forwarded-Host $server_name;
        
        }
    }
}

When you are done modifying the configuration, press CTRL+X and confirm with Y to save and exit. You will need to restart Nginx for changes to come into effect.

Run the following to restart Nginx:

sudo service nginx stop
sudo service nginx start

And that’s it! After connecting your application server with Nginx, you can now visit it by going to your droplet’s IP address using your favourite browser.

http://[your droplet's IP adde.]/

# Hello world!

Further Reading

---

If you would like to learn more about Python web-application deployments, you are recommended to check out our following articles on the subject for a better general understanding:

• How to Deploy Python WSGI Applications Using uWSGI Web Server with Nginx

• How to Deploy Python WSGI Apps Using Gunicorn HTTP Server Behind Nginx

• A Comparison of Web Servers for Python Based Web Applications