How To Structure Large Flask Applications

Introduction

---

There are many methods and conventions for structuring Python web applications. Although certain frameworks are shipped with tools (for scaffolding) to automate – and ease – the task (and the headaches), almost all solutions rely on packaging / modularizing applications as the codebase gets distributed [logically] across related files and folders.

The minimalist web application development framework Flask, has its own - blueprints.

In this DigitalOcean article, we are going to see how to create an application directory, and structure it to work with re-usable components created with Flask’s blueprints. These pieces allow (and simplify) the maintenance and the development of application components greatly.

Glossary

---

1. Flask: The Minimalist Application Development Framework

---

2. Our Choices In This Article

---

3. Preparing The System For Flask

---

1. Prepare The Operating System
2. Setting up Python, pip and virtualenv

4. Structuring The Application Directory

---

1. Creating Application Folder
2. Creating A Virtual Environment
3. Creating Application Files
4. Installing Flask

5. Working With Modules And Blueprints (Components)

---

1. Module Basics
2. Module Templates

6. Creating The Application (run.py, init.py, etc.)

---

1. Edit run.py using nano
2. Edit config.py using nano

7. Creating A Module / Component

---

1. Step 1: Structuring The Module
2. Step 2: Define The Module Data Model(s)
3. Step 3: Define Module Forms
4. Step 4: Define Application Controllers (Views)
5. Step 5: Set Up The Application in “app/init.py”
6. Step 6: Create The Templates
7. Step 7: See Your Module In Action

Flask: The Minimalist Application Development Framework

---

Flask is a minimalist (or micro) framework which refrains from imposing the way critical things are handled. Instead, Flask allows the developers to use the tools they desire and are familiar with. For this purpose, it comes with its own extensions index and a good amount of tools already exist to handle pretty much everything from log-ins to logging.

It is not a strictly “conventional” framework and relies partially on configuration files, which frankly make many things easier when it comes to getting started and keeping things in check.

Our Choices In This Article

---

As we have just been over in the previous section, Flask-way of doing things involves using the tools you are most comfortable with. In our article, we will be using – perhaps – the most common (and sensible) of choices in terms of extensions and libraries (i.e. database extraction layer). These choices will involve:

• SQLAlchemy (via Flask-SQLAlchemy)

• WTForms (via Flask-WTF)

Flask-SQLAlchemy

---

Adds SQLAlchemy support to Flask. Quick and easy.

This is an approved extension.

Author: Armin Ronacher
PyPI Page: Flask-SQLAlchemy
Documentation: Read docs @ packages.python.org
On Github: [mitsuhiko/flask-sqlalchemy](https://github.com/mitsuhiko/flask-sqlalchemy)

Flask-WTF

---

Flask-WTF offers simple integration with WTForms. This integration includes optional CSRF handling for greater security.

This is an approved extension.

Author: Anthony Ford (created by Dan Jacob)
PyPI Page: Flask-WTF
Documentation: Read docs @ packages.python.org
On Github: [ajford/flask-wtf](https://github.com/mitsuhiko/flask-wtf)

Preparing The System For Flask

---

Before we begin structuring a large Flask application, let’s prepare our system and download (and install) Flask distribution.

Note: We will be working on a freshly instantiated droplet running the latest version of available operating systems (i.e. Ubuntu 13). You are highly advised to test everything on a new system as well - especially if you are actively serving clients.

Prepare The Operating System

---

In order to have a stable server, we must have all relevant tools and libraries up-to-date and well maintained.

To ensure that we have the latest available versions of default applications, let’s begin with updates.

Run the following for Debian Based Systems (i.e. Ubuntu, Debian):

aptitude    update
aptitude -y upgrade

To get the necessary development tools, install “build-essential” using the following command:

aptitude install -y build-essential python-dev python2.7-dev

Setting up Python, pip and virtualenv

---

On Ubuntu and Debian, a recent version of Python interpreter - which you can use - comes by default. It leaves us with only a limited number of additional packages to install:

• python-dev (development tools)

• pip (to manage packages)

• virtualenv (to create isolated, virtual

Note: Instructions given here are kept brief. To learn more, check out our how-to article on pip and virtualenv: Common Python Tools: Using virtualenv, Installing with Pip, and Managing Packages.

pip

---

pip is a package manager which will help us to install the application packages that we need.

Run the following commands to install pip:

curl https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py | python -
curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python -
export PATH="/usr/local/bin:$PATH"

virtualenv

---

It is best to contain a Python application within its own environment together with all of its dependencies. An environment can be best described (in simple terms) as an isolated location (a directory) where everything resides. For this purpose, a tool called virtualenv is used.

Run the following to install virtualenv using pip:

sudo pip install virtualenv

Structuring The Application Directory

---

We will use the exemplary name of LargeApp as our application folder. Inside, we are going to have a virtual environment (i.e. env) alongside the application package (i.e. app) and some other files such as “run.py” for running a test (development) server and “config.py” for keeping the Flask configurations.

The structure - which is given as an example below - is highly extensible and it is built to make use of all helpful tools Flask and other libraries offer. Do not be afraid when you see it, as we explain everything step by step by constructing it all.

Target example structure:

~/LargeApp
    |-- run.py
    |-- config.py
    |__ /env             # Virtual Environment
    |__ /app             # Our Application Module
         |-- __init__.py
         |-- /module_one
             |-- __init__.py
             |-- controllers.py
             |-- models.py                
         |__ /templates
             |__ /module_one
                 |-- hello.html
         |__ /static
         |__ ..
         |__ .
    |__ ..
    |__ .

Creating Application Folders

---

Let’s start with creating the main folders we need.

Run the following commands successively to perform the task:

mkdir ~/LargeApp
mkdir ~/LargeApp/app
mkdir ~/LargeApp/app/templates
mkdir ~/LargeApp/app/static    

Our current structure:

~/LargeApp
    |__ /app             # Our Application Module
         |__ /templates
         |__ /static

Creating A Virtual Environment

---

Using a virtual environment brings with it a ton of benefits. You are highly suggested to use a new virtual environment for each one of of your applications. Keeping the virtualenv folder inside your application’s is a good way of keeping things in order and tidy.

Run the following to create a new virtual environment with pip installed.

cd         ~/LargeApp
virtualenv env

Creating Application Files

---

In this step, we will form the basic application files before moving on to working with modules and blueprints.

Run the following to create basic application files:

touch ~/LargeApp/run.py
touch ~/LargeApp/config.py
touch ~/LargeApp/app/__init__.py

Our current structure:

~/LargeApp
    |-- run.py
    |-- config.py
    |__ /env             # Virtual Environment
    |__ /app             # Our Application Module
         |-- __init__.py
         |__ /templates
         |__ /static

Installing Flask And Application Dependencies

---

Once we have everything in place, to begin our development with Flask, let’s download and install it using pip.

Run the following to install Flask inside the virtual environment env.

cd ~/LargeApp
env/bin/pip install flask
env/bin/pip install flask-sqlalchemy
env/bin/pip install flask-wtf

Note: Here we are downloading and installing Flask without activating the virtual environment. However, given that we are using the pip from the environment itself, it achieves the same task. If you are working with an activated environment, you can just use pip instead.

And that’s it! We are now ready to build a larger Flask application modularized using blueprints.

Working With Modules And Blueprints (Components)

---

Module Basics

---

At this point, we have both our application structure set up and its dependencies downloaded and ready.

Our goal is to modularize (i.e. create re-usable components with Flask’s blueprints) all related modules that can be logically grouped.

An example for this can be an authentication system. Having all its views, controllers, models and helpers in one place, set up in a way that allows reusability makes this kind of structuring a great way for maintaining applications whilst increasing productivity.

Target example module (component) structure (inside /app):

# Our module example here is called *mod_auth*
# You can name them as you like as long as conventions are followed

/mod_auth
    |-- __init__.py
    |-- controllers.py
    |-- models.py
    |-- ..
    |-- .

Module Templates

---

To support modularizing to-the-max, we will structure the “templates” folder to follow the above convention and contain a new folder – with the same or a similar, related name as the module – to contain its template files.

Target example templates directory structure (inside LargeApp):

/templates
    |-- 404.html
    |__ /auth
         |-- signin.html
         |-- signup.html
         |-- forgot.html
         |-- ..
         |-- .

Creating The Application

---

In this section, we will continue on the previous steps and start with actual coding of our application before moving onto creating our first modularized component (using blueprints): mod_auth for handling all authentication related procedures (i.e. signing-in, signing-up, etc).

Edit “run.py” using nano

---

nano ~/LargeApp/run.py

Place the contents:

# Run a test server.
from app import app
app.run(host='0.0.0.0', port=8080, debug=True)

Save and exit using CTRL+X and confirm with with Y.

Edit “config.py” using nano

---

nano ~/LargeApp/config.py

Place the contents:

# Statement for enabling the development environment
DEBUG = True

# Define the application directory
import os
BASE_DIR = os.path.abspath(os.path.dirname(__file__))  

# Define the database - we are working with
# SQLite for this example
SQLALCHEMY_DATABASE_URI = 'sqlite:///' + os.path.join(BASE_DIR, 'app.db')
DATABASE_CONNECT_OPTIONS = {}

# Application threads. A common general assumption is
# using 2 per available processor cores - to handle
# incoming requests using one and performing background
# operations using the other.
THREADS_PER_PAGE = 2

# Enable protection agains *Cross-site Request Forgery (CSRF)*
CSRF_ENABLED     = True

# Use a secure, unique and absolutely secret key for
# signing the data. 
CSRF_SESSION_KEY = "secret"

# Secret key for signing cookies
SECRET_KEY = "secret"

Save and exit using CTRL+X and confirm with with Y.

Creating A Module / Component

---

This section is the first major step that defines the core of this article. Here, we will see how to use Flask’s blueprints to create a module (i.e. a component).

What’s brilliant about this is the offered portability and reusability of your code, combined with ease of maintenance - for which you will be thankful in the future as often it is quite bit of a struggle to come back and understand things as they were left.

Step 1: Structuring The Module

---

As we have set out to do, let us create our first module’s (mod_auth) directories and files to start working on them.

# Create the module directory inside the *app* module
mkdir ~/LargeApp/app/mod_auth

# Create where module's templates will reside
mkdir ~/LargeApp/app/templates/auth

# Create __init__.py to set the directory as a Python module
touch ~/LargeApp/app/mod_auth/__init__.py

# Create module's controllers and models etc.
touch ~/LargeApp/app/mod_auth/controllers.py
touch ~/LargeApp/app/mod_auth/models.py
touch ~/LargeApp/app/mod_auth/forms.py

# Create module's templates
touch ~/LargeApp/app/templates/auth/signin.html

# Create a HTTP 404 Error page
touch ~/LargeApp/app/templates/404.html

After these operations, this is how the folder structure should look like:

~/LargeApp
    |-- run.py
    |-- config.py
    |__ /env             # Virtual Environment
    |__ /app             # Our Application Module
         |-- __init__.py
         |-- /mod_auth   # Our first module, mod_auth
             |-- __init__.py
             |-- controllers.py
             |-- models.py
             |-- forms.py
         |__ /templates
             |-- 404.html
             |__ /auth
                 |-- signin.html
         |__ /static

Step 2: Define The Module Data Model(s)

---

nano ~/LargeApp/app/mod_auth/models.py

Place the below self-explanatory - exemplary - contents:

# Import the database object (db) from the main application module
# We will define this inside /app/__init__.py in the next sections.
from app import db

# Define a base model for other database tables to inherit
class Base(db.Model):

    __abstract__  = True

    id            = db.Column(db.Integer, primary_key=True)
    date_created  = db.Column(db.DateTime,  default=db.func.current_timestamp())
    date_modified = db.Column(db.DateTime,  default=db.func.current_timestamp(),
                                           onupdate=db.func.current_timestamp())

# Define a User model
class User(Base):

    __tablename__ = 'auth_user'

    # User Name
    name    = db.Column(db.String(128),  nullable=False)

    # Identification Data: email & password
    email    = db.Column(db.String(128),  nullable=False,
                                            unique=True)
    password = db.Column(db.String(192),  nullable=False)

    # Authorisation Data: role & status
    role     = db.Column(db.SmallInteger, nullable=False)
    status   = db.Column(db.SmallInteger, nullable=False)

    # New instance instantiation procedure
    def __init__(self, name, email, password):

        self.name     = name
        self.email    = email
        self.password = password

    def __repr__(self):
        return '<User %r>' % (self.name)                        

Save and exit using CTRL+X and confirm with with Y.

Step 3: Define Module Forms

---

nano ~/LargeApp/app/mod_auth/forms.py

Place the below self-explanatory - exemplary - contents:

# Import Form and RecaptchaField (optional)
from flask.ext.wtf import Form # , RecaptchaField

# Import Form elements such as TextField and BooleanField (optional)
from wtforms import TextField, PasswordField # BooleanField

# Import Form validators
from wtforms.validators import Required, Email, EqualTo


# Define the login form (WTForms)

class LoginForm(Form):
    email    = TextField('Email Address', [Email(),
                Required(message='Forgot your email address?')])
    password = PasswordField('Password', [
                Required(message='Must provide a password. ;-)')])

Save and exit using CTRL+X and confirm with with Y.

Step 4: Define Application Controllers (Views)

---

nano ~/LargeApp/app/mod_auth/controllers.py

Place the below self-explanatory - exemplary - contents:

# Import flask dependencies
from flask import Blueprint, request, render_template, \
                  flash, g, session, redirect, url_for

# Import password / encryption helper tools
from werkzeug import check_password_hash, generate_password_hash

# Import the database object from the main app module
from app import db

# Import module forms
from app.mod_auth.forms import LoginForm

# Import module models (i.e. User)
from app.mod_auth.models import User

# Define the blueprint: 'auth', set its url prefix: app.url/auth
mod_auth = Blueprint('auth', __name__, url_prefix='/auth')

# Set the route and accepted methods
@mod_auth.route('/signin/', methods=['GET', 'POST'])
def signin():

    # If sign in form is submitted
    form = LoginForm(request.form)

    # Verify the sign in form
    if form.validate_on_submit():

        user = User.query.filter_by(email=form.email.data).first()

        if user and check_password_hash(user.password, form.password.data):

            session['user_id'] = user.id

            flash('Welcome %s' % user.name)

            return redirect(url_for('auth.home'))

        flash('Wrong email or password', 'error-message')

    return render_template("auth/signin.html", form=form)

Save and exit using CTRL+X and confirm with with Y.

Step 5: Set Up The Application in “app/init.py”

---

nano ~/LargeApp/app/__init__.py

Place the contents:

# Import flask and template operators
from flask import Flask, render_template

# Import SQLAlchemy
from flask.ext.sqlalchemy import SQLAlchemy

# Define the WSGI application object
app = Flask(__name__)

# Configurations
app.config.from_object('config')

# Define the database object which is imported
# by modules and controllers
db = SQLAlchemy(app)

# Sample HTTP error handling
@app.errorhandler(404)
def not_found(error):
    return render_template('404.html'), 404

# Import a module / component using its blueprint handler variable (mod_auth)
from app.mod_auth.controllers import mod_auth as auth_module

# Register blueprint(s)
app.register_blueprint(auth_module)
# app.register_blueprint(xyz_module)
# ..

# Build the database:
# This will create the database file using SQLAlchemy
db.create_all()

Save and exit using CTRL+X and confirm with with Y.

Step 6: Create The Templates

---

nano ~/LargeApp/app/templates/auth/signin.html

Place the contents:

{% macro render_field(field, placeholder=None) %}
{% if field.errors %}
<div>
{% elif field.flags.error %}
<div>
{% else %}
<div>
{% endif %}
    {% set css_class = 'form-control ' + kwargs.pop('class', '') %}
    {{ field(class=css_class, placeholder=placeholder, **kwargs) }}
</div>
{% endmacro %}

<div>
  <div>
    <legend>Sign in</legend>
    {% with errors = get_flashed_messages(category_filter=["error"]) %}
    {% if errors %}
    <div>
    {% for error in errors %}
    {{ error }}<br>
    {% endfor %}
    </div>
    {% endif %}
    {% endwith %}

    {% if form.errors %}
    <div>
    {% for field, error in form.errors.items() %}
    {% for e in error %}
    {{ e }}<br>
    {% endfor %}
    {% endfor %}
    </div>
    {% endif %}
    <form method="POST" action="." accept-charset="UTF-8" role="form">
      {{ form.csrf_token }}
      {{ render_field(form.email, placeholder="Your Email Address",
                                  autofocus="") }}
      {{ render_field(form.password, placeholder="Password") }}
      <div>
      <label>
        <input type="checkbox" name="remember" value="1"> Remember Me
      </label>
      <a role="button" href="">Forgot your password?</a><span class="clearfix"></span>
      </div>
      <button type="submit" name="submit">Sign in</button>
    </form>  
  </div>
</div>

Save and exit using CTRL+X and confirm with with Y.

Note: This template file is a very simple and incomplete example that has just been created for demonstration purposes. You are highly encouraged to read the Jinja2 documentation and use base files to build your website’s templates.

Step 7: See Your Module In Action

---

After having created our first module, it is time to see everything in action.

Run a development server using the run.py:

cd ~/LargeApp
env/bin/python run.py

This will initiate a development (i.e. testing) server hosted at port 8080.

Visit the module by going to the URL:

http://[your droplet's IP]/auth/signin

Although you will not be able to login, you can see it in action by entering some exemplary data or by testing its validators.

<div class=“author”>Submitted by: <a href=“https://twitter.com/ostezer”>O.S. Tezer</a></div>

env/bin/python run.py
Traceback (most recent call last):
  File "run.py", line 2, in <module>
    from app import app
  File "/home/arnold/labs/flask/yauler/app/__init__.py", line 23, in <module>
    from app.mod_auth.controllers import mod_auth as auth_module
ImportError: No module named mod_auth.controllers
[1]    12190 exit 1     env/bin/python run.py

ImportError: cannot import name 'db' from 'app'