How To Perform Unit Testing in Flask

Introduction

Testing is essential to the software development process, ensuring that code behaves as expected and is defect-free. In Python, pytest is a popular testing framework that offers several advantages over the standard unit test module, which is a built-in Python testing framework and is part of the standard library. pytest includes a simpler syntax, better output, powerful fixtures, and a rich plugin ecosystem. This tutorial will guide you through setting up a Flask application, integrating pytest fixtures, and writing unit tests using pytest.

Prerequisites

Before you begin, you’ll need the following:

• A server running Ubuntu and a non-root user with sudo privileges and an active firewall. For guidance on how to set this up, please choose your distribution from this list and follow our initial server setup guide. Please ensure to work with a supported version of Ubuntu.

• Familiarity with the Linux command line. You can visit this guide on Linux command line primer.

• A basic understanding of Python programming and pytest testing framework in Python. You can refer to our tutorial on PyTest Python Testing Framework to learn more about pytest.

• Python 3.7 or higher installed on your Ubuntu system. To learn how to run a Python script on Ubuntu, you can refer to our tutorial on How to run a Python script on Ubuntu.

Why pytest is a Better Alternative to unittest

pytest offers several advantages over the built-in unittest framework:

• Pytest allows you to write tests with less boilerplate code, using simple assert statements instead of the more verbose methods required by unittest.

• It provides more detailed and readable output, making it easier to identify where and why a test failed.

• Pytest fixtures allow for more flexible and reusable test setups than unittest’s setUp and tearDown methods.

• It makes it easy to run the same test function with multiple sets of input, which is not as straightforward in unittest.

• Pytest has a rich collection of plugins that extend its functionality, from code coverage tools to parallel test execution.

• It automatically discovers test files and functions that match its naming conventions, saving time and effort in managing test suites.

Given these benefits, pytest is often the preferred choice for modern Python testing. Let’s set up a Flask application and write unit tests using pytest.

Step 1 - Setting Up the Environment

Ubuntu 24.04 ships Python 3 by default. Open the terminal and run the following command to double-check the Python 3 installation:

root@ubuntu:~# python3 --version
Python 3.12.3

If Python 3 is already installed on your machine, the above command will return the current version of Python 3 installation. In case it is not installed, you can run the following command and get the Python 3 installation:

root@ubuntu:~# sudo apt install python3

Next, you need to install the pip package installer on your system:

root@ubuntu:~# sudo apt install python3-pip

Once pip is installed, let’s install Flask.

Step 2 - Create a Flask Application

Let’s start by creating a simple Flask application. Create a new directory for your project and navigate into it:

root@ubuntu:~# mkdir flask_testing_app
root@ubuntu:~# cd flask_testing_app

Now, let’s create and activate a virtual environment to manage dependencies:

root@ubuntu:~# python3 -m venv venv
root@ubuntu:~# source venv/bin/activate

Install Flask using pip:

root@ubuntu:~# pip install Flask

Now, let’s create a simple Flask application. Create a new file named app.py and add the following code:

from flask import Flask, jsonify

app = Flask(__name__)

@app.route('/')
def home():
    return jsonify(message="Hello, Flask!")

@app.route('/about')
def about():
    return jsonify(message="This is the About page")

@app.route('/multiply/<int:x>/<int:y>')
def multiply(x, y):
    result = x * y
    return jsonify(result=result)

if __name__ == '__main__':
    app.run(debug=True)

This application has three routes:

• /: Returns a simple “Hello, Flask!” message.
• /about: Returns a simple “This is the About page” message.
• /multiply/<int:x>/<int:y>: Multiplies two integers and returns the result.

To run the application, execute the following command:

root@ubuntu:~# flask run

output
 * Serving Flask app "app" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

From the above output you can notice that the server is running on http://127.0.0.1 and listening on port 5000. Open another Ubuntu Console and execute the below curl commands one by one:

• GET: curl http://127.0.0.1:5000/
• GET: curl http://127.0.0.1:5000/about
• GET: curl http://127.0.0.1:5000/multiply/10/20

Let’s understand what these GET requests do:

1. curl http://127.0.0.1:5000/: This sends a GET request to the root route (‘/’) of our Flask application. The server responds with a JSON object containing the message “Hello, Flask!”, demonstrating the basic functionality of our home route.

2. curl http://127.0.0.1:5000/about: This sends a GET request to the /about route. The server responds with a JSON object containing the message “This is the About page”. This shows that our route is functioning correctly.

3. curl http://127.0.0.1:5000/multiply/10/20: This sends a GET request to the /multiply route with two parameters: 10 and 20. The server multiplies these numbers and responds with a JSON object containing the result (200). This demonstrates that our multiply route can correctly process URL parameters and perform calculations.

These GET requests allow us to interact with our Flask application’s API endpoints, retrieving information or triggering actions on the server without modifying any data. They’re useful for fetching data, testing endpoint functionality, and verifying that our routes are responding as expected.

Let’s see each of these GET requests in action:

root@ubuntu:~# curl http://127.0.0.1:5000/

Output
{"message":"Hello, Flask!"}

root@ubuntu:~# curl http://127.0.0.1:5000/about

Output
{"message":"This is the About page"}

root@ubuntu:~# curl http://127.0.0.1:5000/multiply/10/20

Output
{"result":200}

Step 3 - Installing pytest and Writing Your First Test

Now that you have a basic Flask application, let’s install pytest and write some unit tests.

Install pytest using pip:

root@ubuntu:~# pip install pytest

Create a tests directory to store your test files:

root@ubuntu:~# mkdir tests

Now, let’s create a new file named test_app.py and add the following code:

# Import sys module for modifying Python's runtime environment
import sys
# Import os module for interacting with the operating system
import os

# Add the parent directory to sys.path
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

# Import the Flask app instance from the main app file
from app import app 
# Import pytest for writing and running tests
import pytest

@pytest.fixture
def client():
    """A test client for the app."""
    with app.test_client() as client:
        yield client

def test_home(client):
    """Test the home route."""
    response = client.get('/')
    assert response.status_code == 200
    assert response.json == {"message": "Hello, Flask!"}

def test_about(client):
    """Test the about route."""
    response = client.get('/about')
    assert response.status_code == 200
    assert response.json == {"message": "This is the About page"}

def test_multiply(client):
    """Test the multiply route with valid input."""
    response = client.get('/multiply/3/4')
    assert response.status_code == 200
    assert response.json == {"result": 12}

def test_multiply_invalid_input(client):
    """Test the multiply route with invalid input."""
    response = client.get('/multiply/three/four')
    assert response.status_code == 404

def test_non_existent_route(client):
    """Test for a non-existent route."""
    response = client.get('/non-existent')
    assert response.status_code == 404

Let’s break down the functions in this test file:

1. @pytest.fixture def client(): This is a pytest fixture that creates a test client for our Flask app. It uses the app.test_client() method to create a client that can send requests to our app without running the actual server. The yield statement allows the client to be used in tests and then properly closed after each test.

2. def test_home(client): This function tests the home route (/) of our app. It sends a GET request to the route using the test client, then asserts that the response status code is 200 (OK) and that the JSON response matches the expected message.

3. def test_about(client): Similar to test_home, this function tests the about route (/about). It checks for a 200 status code and verifies the JSON response content.

4. def test_multiply(client): This function tests the multiply route with valid input (/multiply/3/4). It checks that the status code is 200 and that the JSON response contains the correct result of the multiplication.

5. def test_multiply_invalid_input(client): This function tests the multiply route with invalid input (multiply/three/four). It checks that the status code is 404 (Not Found), which is the expected behavior when the route can’t match the string inputs to the required integer parameters.

6. def test_non_existent_route(client): This function tests the behavior of the app when a non-existent route is accessed. It sends a GET request to /non-existent, which is not defined in our Flask app. The test asserts that the response status code is 404 (Not Found), ensuring that our app correctly handles requests to undefined routes.

These tests cover the basic functionality of our Flask app, ensuring that each route responds correctly to valid inputs and that the multiply route handles invalid inputs appropriately. By using pytest, we can easily run these tests to verify that our app is working as expected.

Step 4 - Running the Tests

To run the tests, execute the following command:

root@ubuntu:~# pytest

By default, the pytest discovery process will recursively scan the current folder and its subfolders for files starting with names either “test_” or ending with “_test”. Tests located in those files are then executed. You should see output similar to:

Output
platform linux -- Python 3.12.3, pytest-8.3.2, pluggy-1.5.0
rootdir: /home/user/flask_testing_app
collected 5 items                                                                                                                

tests/test_app.py ....                                                                                                     [100%]

======================================================= 5 passed in 0.19s ========================================================

This indicates that all tests have passed successfully.

Step 5: Using Fixtures in pytest

Fixtures are functions that are used to provide data or resources to tests. They can be used to set up and tear down test environments, load data, or perform other setup tasks. In pytest, fixtures are defined using the @pytest.fixture decorator.

Here’s how to enhance the existing fixture. Update the client fixture to use setup and teardown logic:

@pytest.fixture
def client():
    """Set up a test client for the app with setup and teardown logic."""
    print("\nSetting up the test client")
    with app.test_client() as client:
        yield client  # This is where the testing happens
    print("Tearing down the test client")

def test_home(client):
    """Test the home route."""
    response = client.get('/')
    assert response.status_code == 200
    assert response.json == {"message": "Hello, Flask!"}

def test_about(client):
    """Test the about route."""
    response = client.get('/about')
    assert response.status_code == 200
    assert response.json == {"message": "This is the About page"}

def test_multiply(client):
    """Test the multiply route with valid input."""
    response = client.get('/multiply/3/4')
    assert response.status_code == 200
    assert response.json == {"result": 12}

def test_multiply_invalid_input(client):
    """Test the multiply route with invalid input."""
    response = client.get('/multiply/three/four')
    assert response.status_code == 404

def test_non_existent_route(client):
    """Test for a non-existent route."""
    response = client.get('/non-existent')
    assert response.status_code == 404

This setup adds print statements to demonstrate the setup and teardown phases in the test output. These can be replaced with actual resource management code if needed.

Let’s try to run the tests again:

root@ubuntu:~# pytest -vs

The -v flag increases verbosity, and the -s flag allows print statements to be displayed in the console output.

You should see the following output:

Output
platform linux -- Python 3.12.3, pytest-8.3.2, pluggy-1.5.0
rootdir: /home/user/flask_testing_app 
cachedir: .pytest_cache      

collected 5 items                                                                                          

tests/test_app.py::test_home 
Setting up the test client
PASSED
Tearing down the test client

tests/test_app.py::test_about 
Setting up the test client
PASSED
Tearing down the test client

tests/test_app.py::test_multiply 
Setting up the test client
PASSED
Tearing down the test client

tests/test_app.py::test_multiply_invalid_input 
Setting up the test client
PASSED
Tearing down the test client

tests/test_app.py::test_non_existent_route 
Setting up the test client
PASSED
Tearing down the test client

============================================ 5 passed in 0.35s =============================================

Step 6: Adding a Failure Test Case

Let’s add a failure test case to the existing test file. Modify the test_app.py file and add the below function towards the end for a failing test case for an incorrect result:

def test_multiply_edge_cases(client):
    """Test the multiply route with edge cases to demonstrate failing tests."""
    # Test with zero
    response = client.get('/multiply/0/5')
    assert response.status_code == 200
    assert response.json == {"result": 0}

    # Test with large numbers (this might fail if not handled properly)
    response = client.get('/multiply/1000000/1000000')
    assert response.status_code == 200
    assert response.json == {"result": 1000000000000}

    # Intentional failing test: incorrect result
    response = client.get('/multiply/2/3')
    assert response.status_code == 200
    assert response.json == {"result": 7}, "This test should fail to demonstrate a failing case"

Let’s break down the test_multiply_edge_cases function and explain what each part does:

1. Test with zero: This test checks if the multiply function correctly handles multiplication by zero. We expect the result to be 0 when multiplying any number by zero. This is an important edge case to test because some implementations might have issues with zero multiplication.

2. Test with large numbers: This test verifies if the multiply function can handle large numbers without overflow or precision issues. We’re multiplying two one million values, expecting a result of one trillion. This test is crucial because it checks the upper limits of the function’s capability. Note that this might fail if the server’s implementation doesn’t handle large numbers properly, which could indicate a need for big number libraries or a different data type.

3. Intentional failing test: This test is deliberately set up to fail. It checks if 2 * 3 equals 7, which is incorrect. This test aims to demonstrate how a failing test looks in the test output. This helps in understanding how to identify and debug failing tests, which is an essential skill in test-driven development and debugging processes.

By including these edge cases and an intentional failure, you’re testing not only the basic functionality of your multiply route but also its behavior under extreme conditions and its error reporting capabilities. This approach to testing helps ensure the robustness and reliability of our application.

Let’s try to run the tests again:

root@ubuntu:~# pytest -vs

You should see the following output:

Output
platform linux -- Python 3.12.3, pytest-8.3.2, pluggy-1.5.0
rootdir: /home/user/flask_testing_app 
cachedir: .pytest_cache      
  
collected 6 items                                                                                                                           

tests/test_app.py::test_home 
Setting up the test client
PASSED
Tearing down the test client

tests/test_app.py::test_about 
Setting up the test client
PASSED
Tearing down the test client

tests/test_app.py::test_multiply 
Setting up the test client
PASSED
Tearing down the test client

tests/test_app.py::test_multiply_invalid_input 
Setting up the test client
PASSED
Tearing down the test client

tests/test_app.py::test_non_existent_route 
Setting up the test client
PASSED
Tearing down the test client

tests/test_app.py::test_multiply_edge_cases 
Setting up the test client
FAILED
Tearing down the test client


================================================================= FAILURES ==================================================================
_________________________________________________________ test_multiply_edge_cases __________________________________________________________

client = <FlaskClient <Flask 'app'>>

    def test_multiply_edge_cases(client):
        """Test the multiply route with edge cases to demonstrate failing tests."""
        # Test with zero
        response = client.get('/multiply/0/5')
        assert response.status_code == 200
        assert response.json == {"result": 0}
    
        # Test with large numbers (this might fail if not handled properly)
        response = client.get('/multiply/1000000/1000000')
        assert response.status_code == 200
        assert response.json == {"result": 1000000000000}
    
        # Intentional failing test: incorrect result
        response = client.get('/multiply/2/3')
        assert response.status_code == 200
>       assert response.json == {"result": 7}, "This test should fail to demonstrate a failing case"
E       AssertionError: This test should fail to demonstrate a failing case
E       assert {'result': 6} == {'result': 7}
E         
E         Differing items:
E         {'result': 6} != {'result': 7}
E         
E         Full diff:
E           {
E         -     'result': 7,...
E         
E         ...Full output truncated (4 lines hidden), use '-vv' to show

tests/test_app.py:61: AssertionError
========================================================== short test summary info ==========================================================
FAILED tests/test_app.py::test_multiply_edge_cases - AssertionError: This test should fail to demonstrate a failing case
======================================================== 1 failed, 5 passed in 0.32s ========================================================

The failure message above indicates that the test test_multiply_edge_cases in the tests/test_app.py file failed. Specifically, the last assertion in this test function caused the failure.

This intentional failure is useful for demonstrating how test failures are reported and what information is provided in the failure message. It shows the exact line where the failure occurred, the expected and actual values, and the difference between the two.

In a real-world scenario, you would fix the code to make the test pass or adjust the test if the expected result was incorrect. However, in this case, the failure is intentional for educational purposes.

Conclusion

In this tutorial, we covered how to set up unit tests for a Flask application using pytest, integrated pytest fixtures, and demonstrated what a test failure looks like. By following these steps, you can ensure your Flask applications are reliable and maintainable, minimizing bugs and enhancing code quality.

You can refer to Flask and Pytest official documentation to learn more.