How To Deploy an Advanced PHP Application Using Ansible on Ubuntu 14.04

Introduction

This tutorial is the second in a series about deploying PHP applications using Ansible on Ubuntu 14.04. The first tutorial covers the basic steps for deploying an application, and is a starting point for the steps outlined in this tutorial.

In this tutorial we will cover setting up SSH keys to support code deployment/publishing tools, configuring the system firewall, provisioning and configuring the database (including the password!), and setting up task schedulers (crons) and queue daemons. The goal at the end of this tutorial is for you to have a fully working PHP application server with the aforementioned advanced configuration.

Like the last tutorial, we will be using the Laravel framework as our example PHP application. However, these instructions can be easily modified to support other frameworks and applications if you already have your own.

Prerequisites

This tutorial follows on directly from the end of the first tutorial in the series, and all of the configuration and files generated for that tutorial are required. If you haven’t completed that tutorial yet, please do so first before continuing with this tutorial.

Step 1 — Switching the Application Repository

In this step, we will update the Git repository to a slightly customized example repository.

Because the default Laravel installation doesn’t require the advanced features that we will be setting up in this tutorial, we will be switching the existing repository from the standard repository to an example repository with some debugging code added, just to show when things are working. The repository we will use is located at https://github.com/do-community/do-ansible-adv-php.

If you haven’t done so already, change directories into ansible-php from the previous tutorial.

cd ~/ansible-php/

Open up our existing playbook for editing.

nano php.yml

Find and update the “Clone git repository” task, so it looks like this.

- name: Clone git repository
  git: >
    dest=/var/www/laravel
    repo=https://github.com/do-community/do-ansible-adv-php
    update=yes
    version=example
  sudo: yes
  sudo_user: www-data
  register: cloned

Save and run the playbook.

ansible-playbook php.yml --ask-sudo-pass

When it has finished running, visit your server in your web browser (i.e. http://your_server_ip/). You should see a message that says “could not find driver”.

This means we have successfully swapped out the default repository for our example repository, but the application cannot connect to the database. This is what we expect to see here, and we will install and set up the database later in the tutorial.

Step 2 — Setting up SSH Keys for Deployment

In this step, we will set up SSH keys that can be used for application code deployment scripts.

While Ansible is great for maintaining configuration and setting up servers and applications, tools like Envoy and Rocketeer are often used to push code changes onto your server and run application commands remotely. Most of these tools require an SSH connection that can access the application installation directly. In our case, this means we need to configure SSH keys for the www-data user.

We will need the public key file for the user you wish to push your code from. This file is typically found at ~/.ssh/id_rsa.pub. Copy that file into the ansible-php directory.

cp ~/.ssh/id_rsa.pub ~/ansible-php/deploykey.pub

We can use the Ansible authorized_key module to install our public key within /var/www/.ssh/authorized_keys, which will allow the deployment tools to connect and access our application. The configuration only needs to know where the key is, using a lookup, and the user the key needs to be installed for (www-data in our case).

- name: Copy public key into /var/www
  authorized_key: user=www-data key="{{ lookup('file', 'deploykey.pub') }}"

We also need to set the www-data user’s shell, so we can actually log in. Otherwise, SSH will allow the connection, but there will be no shell presented to the user. This can be done using the user module, and setting the shell to /bin/bash (or your preferred shell).

- name: Set www-data user shell
  user: name=www-data shell=/bin/bash

Now, open up the playbook for editing to add in the new tasks.

nano php.yml

Add the above tasks to your php.yml playbook; the end of the file should match the following. The additions are highlighted in red.

. . .

  - name: Configure nginx
    template: src=nginx.conf dest=/etc/nginx/sites-available/default
    notify:
      - restart php5-fpm
      - restart nginx

  - name: Copy public key into /var/www
    authorized_key: user=www-data key="{{ lookup('file', 'deploykey.pub') }}"

  - name: Set www-data user shell
    user: name=www-data shell=/bin/bash

  handlers:
  
. . .

Save and run the playbook.

ansible-playbook php.yml --ask-sudo-pass

When Ansible finishes, you should be able to SSH in using the www-data user.

ssh www-data@your_server_ip

If you successfully log in, it’s working! You can now log back out by entering logout or pressing CTRL+D.

We won’t need to use that connection for any other steps in this tutorial, but it will be useful if you are setting up other tools, as mentioned above, or for general debugging and application maintenance as required.

Step 3 — Configuring the Firewall

In this step we will configure the firewall on the server to allow only connections for HTTP and SSH.

Ubuntu 14.04 comes with UFW (Uncomplicated Firewall) installed by default, and Ansible supports it with the ufw module. It has a number of powerful features and has been designed to be as simple as possible. It’s perfectly suited for self-contained web servers that only need a couple of ports open. In our case, we want port 80 (HTTP) and port 22 (SSH) open. You may also want port 443 for HTTPS.

The ufw module has a number of different options which perform different tasks. The different tasks we need to perform are:

1. Enable UFW and deny all incoming traffic by default.

2. Open the SSH port but rate limit it to prevent brute force attacks.

3. Open the HTTP port.

This can be done with the following tasks, respectively.

- name: Enable UFW
  ufw: direction=incoming policy=deny state=enabled

- name: UFW limit SSH
  ufw: rule=limit port=ssh

- name: UFW open HTTP
  ufw: rule=allow port=http

As before, open the php.yml file for editing.

nano php.yml

Add the above tasks to the the playbook; the end of the file should match the following.

. . .

  - name: Copy public key into /var/www
    authorized_key: user=www-data key="{{ lookup('file', 'deploykey.pub') }}"

  - name: Set www-data user shell
    user: name=www-data shell=/bin/bash

  - name: Enable UFW
    ufw: direction=incoming policy=deny state=enabled

  - name: UFW limit SSH
    ufw: rule=limit port=ssh

  - name: UFW open HTTP
    ufw: rule=allow port=http

  handlers:
  
. . .

Save and run the playbook.

ansible-playbook php.yml --ask-sudo-pass

When that has successfully completed, you should still be able to connect via SSH (using Ansible) or HTTP to your server; other ports will now be blocked.

You can verify the status of UFW at any time by running this command:

ansible php --sudo --ask-sudo-pass -m shell -a "ufw status verbose"

Breaking down the Ansible command above:

• ansible: Run a raw Ansible task, without a playbook.
• php: Run the task against the hosts in this group.
• --sudo: Run the command as sudo.
• --ask-sudo-pass: Prompt for the sudo password.
• -m shell: Run the shell module.
• -a "ufw status verbose": The options to be passed into the module. Because it is a shell command, we pass the raw command (i.e. ufw status verbose) straight in without any key=value options.

It should return something like this.

your_server_ip | success | rc=0 >>
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22                         LIMIT IN    Anywhere
80                         ALLOW IN    Anywhere
22 (v6)                    LIMIT IN    Anywhere (v6)
80 (v6)                    ALLOW IN    Anywhere (v6)

Step 4 — Installing the MySQL Packages

In this step we will set up a MySQL database for our application to use.

The first step is to ensure that MySQL is installed on our server by simply adding the required packages to the install packages task at the top of our playbook. The packages we need are mysql-server, mysql-client, and php5-mysql. We will also need python-mysqldb so Ansible can communicate with MySQL.

As we are adding packages, we need to restart nginx and php5-fpm to ensure the new packages are usable by the application. In this case, we need MySQL to be available to PHP, so it can connect to the database.

One of the fantastic things about Ansible is that you can modify any of the tasks and re-run your playbook and the changes will be applied. This includes lists of options, like we have with the apt task.

As before, open the php.yml file for editing.

nano php.yml

Find the install packages task, and update it to include the packages above:

. . .

- name: install packages
  apt: name={{ item }} update_cache=yes state=latest
  with_items:
    - git
    - mcrypt
    - nginx
    - php5-cli
    - php5-curl
    - php5-fpm
    - php5-intl
    - php5-json
    - php5-mcrypt
    - php5-sqlite
    - sqlite3
    - mysql-server
    - mysql-client
    - php5-mysql
    - python-mysqldb
  notify:
    - restart php5-fpm
    - restart nginx

. . .

Save and run the playbook:

ansible-playbook php.yml --ask-sudo-pass

Step 5 — Setting up the MySQL Database

In this step we will create a MySQL database for our application.

Ansible can talk directly to MySQL using the mysql_-prefaced modules (e.g. mysql_db, mysql_user). The mysql_db module provides a way to ensure a database with a specific name exists, so we can use a task like this to create the database.

- name: Create MySQL DB
  mysql_db: name=laravel state=present

We also need a valid user account with a known password to allow our application to connect to the database. One approach to this is to generate a password locally and save it in our Ansible playbook, but that is insecure and there is a better way.

We will generate the password using Ansible on the server itself and use it directly where it is needed. To generate a password, we will use the makepasswd command line tool, and ask for a 32-character password. Because makepasswd isn’t default on Ubuntu, we will need to add that to the packages list too.

We will also tell Ansible to remember the output of the command (i.e. the password), so we can use it later in our playbook. However, because Ansible doesn’t know if it has already run a shell command, we’ll also create a file when we run that command. Ansible will check if the file exists, and if so, it will assume the command has already been run and won’t run it again.

The task looks like this:

- name: Generate DB password
  shell: makepasswd --chars=32
  args:
    creates: /var/www/laravel/.dbpw
  register: dbpwd

Next, we need to create the actual MySQL database user with the password we specified. This is done using the mysql_user module, and we can use the stdout option on the variable we defined during the password generation task to get the raw output of the shell command, like this: dbpwd.stdout.

The mysql_user command accepts the name of the user and the privileges required. In our case, we want to create a user called laravel and give them full privileges on the laravel table. We also need to tell the task to only run when the dbpwd variable has changed, which will only be when the password generation task is run.

The task should look like this:

- name: Create MySQL User
  mysql_user: name=laravel password={{ dbpwd.stdout }} priv=laravel.*:ALL state=present
  when: dbpwd.changed

Putting this together, open the php.yml file for editing, so we can add in the above tasks.

nano php.yml

Firstly, find the install packages task, and update it to include the makepasswd package.

. . .

- name: install packages
  apt: name={{ item }} update_cache=yes state=latest
  with_items:
    - git
    - mcrypt
    - nginx
    - php5-cli
    - php5-curl
    - php5-fpm
    - php5-intl
    - php5-json
    - php5-mcrypt
    - php5-sqlite
    - sqlite3
    - mysql-server
    - mysql-client
    - php5-mysql
    - python-mysqldb
    - makepasswd
  notify:
    - restart php5-fpm
    - restart nginx

. . .

Then, add the password generation, MySQL database creation, and user creation tasks at the bottom.

. . .

  - name: UFW limit SSH
    ufw: rule=limit port=ssh

  - name: UFW open HTTP
    ufw: rule=allow port=http

  - name: Create MySQL DB
    mysql_db: name=laravel state=present

  - name: Generate DB password
    shell: makepasswd --chars=32
    args:
      creates: /var/www/laravel/.dbpw
    register: dbpwd

  - name: Create MySQL User
    mysql_user: name=laravel password={{ dbpwd.stdout }} priv=laravel.*:ALL state=present
    when: dbpwd.changed

  handlers:

. . .

Do not run the playbook yet! You may have noticed that although we have created the MySQL user and database, we haven’t done anything with the password. We will cover that in the next step. When using shell tasks within Ansible, it is always important to remember to complete the entire workflow that deals with the output/results of the task before running it to avoid having to manually log in and reset the state.

Step 6 — Configuring the PHP Application for the Database

In this step, we will save the MySQL database password into the .env file for the application.

Like we did in the last tutorial, we will update the .env file to include our newly created database credentials. By default Laravel’s .env file contains these lines:

DB_HOST=localhost
DB_DATABASE=homestead
DB_USERNAME=homestead
DB_PASSWORD=secret

We can leave the DB_HOST line as-is, but will update the other three using the following tasks, which are very similar to the tasks we used in the previous tutorial to set APP_ENV and APP_DEBUG.

- name: set DB_DATABASE
  lineinfile: dest=/var/www/laravel/.env regexp='^DB_DATABASE=' line=DB_DATABASE=laravel

- name: set DB_USERNAME
  lineinfile: dest=/var/www/laravel/.env regexp='^DB_USERNAME=' line=DB_USERNAME=laravel

- name: set DB_PASSWORD
  lineinfile: dest=/var/www/laravel/.env regexp='^DB_PASSWORD=' line=DB_PASSWORD={{ dbpwd.stdout }}
  when: dbpwd.changed

As we did with the MySQL user creation task, we have used the generated password variable (dbpwd.stdout) to populate the file with the password, and have added the when option to ensure it is only run when dbpwd has changed.

Now, because the .env file already existed before we added our password generation task, we will need to save the password to another file. The generation task can look for that file’s existence (which we already set up within the task). We will also use the sudo and sudo_user options to tell Ansible to create the file as the www-data user.

- name: Save dbpw file
  lineinfile: dest=/var/www/laravel/.dbpw line="{{ dbpwd.stdout }}" create=yes state=present
  sudo: yes
  sudo_user: www-data
  when: dbpwd.changed

Open the php.yml file for editing.

nano php.yml

Add the above tasks to the the playbook; the end of the file should match the following.

. . .

  - name: Create MySQL User
    mysql_user: name=laravel password={{ dbpwd.stdout }} priv=laravel.*:ALL state=present
    when: dbpwd.changed

  - name: set DB_DATABASE
    lineinfile: dest=/var/www/laravel/.env regexp='^DB_DATABASE=' line=DB_DATABASE=laravel

  - name: set DB_USERNAME
    lineinfile: dest=/var/www/laravel/.env regexp='^DB_USERNAME=' line=DB_USERNAME=laravel

  - name: set DB_PASSWORD
    lineinfile: dest=/var/www/laravel/.env regexp='^DB_PASSWORD=' line=DB_PASSWORD={{ dbpwd.stdout }}
    when: dbpwd.changed

  - name: Save dbpw file
    lineinfile: dest=/var/www/laravel/.dbpw line="{{ dbpwd.stdout }}" create=yes state=present
    sudo: yes
    sudo_user: www-data
    when: dbpwd.changed

  handlers:

. . .

Again, do not run the playbook yet! We have one more step to complete before we can run the playbook.

Step 7 — Migrating the Database

In this step, we will run the database migrations to set up the database tables.

In Laravel, this is done by running the migrate command (i.e. php artisan migrate --force) within the Laravel directory. Note that we have added the --force flag because the production environment requires it.

The Ansible task to perform this looks like this.

  - name: Run artisan migrate
    shell: php /var/www/laravel/artisan migrate --force
    sudo: yes
    sudo_user: www-data
    when: dbpwd.changed

Now it is time to update our playbook. Open the php.yml file for editing.

nano php.yml

Add the above tasks to the the playbook; the end of the file should match the following.

. . .

  - name: Save dbpw file
    lineinfile: dest=/var/www/laravel/.dbpw line="{{ dbpwd.stdout }}" create=yes   state=present
    sudo: yes
    sudo_user: www-data
    when: dbpwd.changed

  - name: Run artisan migrate
    shell: php /var/www/laravel/artisan migrate --force
    sudo: yes
    sudo_user: www-data
    when: dbpwd.changed

  handlers:

. . .

Finally, we can save and run the playbook.

ansible-playbook php.yml --ask-sudo-pass

When that finishes executing, refresh the page in your browser and you should see a message that says:

Queue: NO
Cron: NO

This means the database is set up correctly and working as expected, but we haven’t yet set up cron tasks or the queue daemon.

Step 8 — Configuring cron Tasks

In this step, we will set up any cron tasks that need to be configured.

Cron tasks are commands that run on a set schedule and can be used to perform any number of tasks for your application, like performing maintenance tasks or sending out email activity updates — essentially anything that needs to be done periodically without manual user intervention. Cron tasks can run as frequently as every minute, or as infrequently as you require.

Laravel comes with an Artisan command called schedule:run by default, which is designed to be run every minute and executes the defined scheduled tasks within the application. This means we only need to add a single cron task, if our application takes advantage of this feature.

Ansible has a cron module with a number of different options that translate directly into the different options you can configure via cron:

• job: The command to execute. Required if state=present.
• minute, hour, day, month, and weekday: The minute, hour, day, month, or day of the week when the job should run, respectively.
• special_time (reboot, yearly, annually, monthly, weekly, daily, hourly): Special time specification nickname.

By default, it will create a task that runs every minute, which is what we want. This means the task we want looks like this:

- name: Laravel Scheduler
  cron: >
    job="run-one php /var/www/laravel/artisan schedule:run 1>> /dev/null 2>&1"
    state=present
    user=www-data
    name="php artisan schedule:run"

The run-one command is a small helper in Ubuntu that ensures the command is only being run once. This means that if a previous schedule:run command is still running, it won’t be run again. This is helpful to avoid a situation where a cron task becomes locked in a loop, and over time, more and more instances of the same task are started until the server runs out of resources.

As before, open the php.yml file for editing.

nano php.yml

Add the above task to the the playbook; the end of the file should match the following.

. . .

  - name: Run artisan migrate
    shell: php /var/www/laravel/artisan migrate --force
    sudo: yes
    sudo_user: www-data
    when: dbpwd.changed

  - name: Laravel Scheduler
    cron: >
      job="run-one php /var/www/laravel/artisan schedule:run 1>> /dev/null 2>&1"
      state=present
      user=www-data
      name="php artisan schedule:run"

  handlers:

. . .

Save and run the playbook:

ansible-playbook php.yml --ask-sudo-pass

Now, refresh the page in your browser. In a minute, it will update to look like this.

Queue: NO
Cron: YES

This means that the cron is working in the background correctly. As part of the example application, there is a cron job that is running every minute updating a status entry in the database so the application knows it is running.

Step 9 — Configuring the Queue Daemon

Like the schedule:run Artisan command from step 8, Laravel also comes with a queue worker that can be started with the queue:work --daemon Artisan command. In this step we will configure the queue daemon worker for Laravel.

Queue workers are similar to cron jobs in that they run tasks in the background. The difference is that the application pushes jobs into the queue, either via actions performed by the user or from tasks scheduled through a cron job. Queue tasks are executed by the worker one at a time, and will be processed on-demand when they are found in the queue. Queue tasks are commonly used for work that takes time to execute, such as sending emails or making API calls to external services.

Unlike the schedule:run command, this isn’t a command that needs to be run every minute. Instead, it needs to run as a daemon in the background constantly. A common way to do this is by using a third party package like supervisord, but that method requires understanding how to configure and manage said system. There is a much simpler way to implement it using cron and the run-one command.

We will create a cron entry to start the queue worker daemon, and use run-one to run it. This means that cron will start the process the first time it runs, and any subsequent cron runs will be ignored by run-one while the worker is running. As soon as the worker stops, run-one will allow the command to run again, and the queue worker will start again. It is an incredibly simple and easy to use method that saves you from needing to learn how to configure and use another tool.

With all of that in mind, we will create another cron task to run our queue worker.

- name: Laravel Queue Worker
  cron: >
    job="run-one php /var/www/laravel/artisan queue:work --daemon --sleep=30 --delay=60 --tries=3 1>> /dev/null 2>&1"
    state=present
    user=www-data
    name="Laravel Queue Worker"

As before, open the php.yml file for editing.

nano php.yml

Add the above task to the the playbook; the end of the file should match the following:

. . .

  - name: Laravel Scheduler
    cron: >
      job="run-one php /var/www/laravel/artisan schedule:run 1>> /dev/null 2>&1"
      state=present
      user=www-data
      name="php artisan schedule:run"

  - name: Laravel Queue Worker
    cron: >
      job="run-one php /var/www/laravel/artisan queue:work --daemon --sleep=30 --delay=60 --tries=3 1>> /dev/null 2>&1"
      state=present
      user=www-data
      name="Laravel Queue Worker"

  handlers:
. . .

Save and run the playbook:

ansible-playbook php.yml --ask-sudo-pass

Like before, refresh the page in your browser. After a minute, it will update to look like this:

Queue: YES
Cron: YES

This means that the queue worker is working in the background correctly. The cron job that we started in the last step pushes a job onto the queue. This job updates the database when it is run to show that it is working.

We now have a working example Laravel application which includes functioning cron jobs and queue workers.

Conclusion

This tutorial covered the some of the more advanced topics when using Ansible for deploying PHP applications. All of the tasks used can be easily modified to suit most PHP applications (depending on their specific requirements), and it should give you a good starting point to set up your own playbooks for your applications.

We have not used a single SSH command as part of this tutorial (apart from checking the www-data user login), and everything — including the MySQL user password — has been set up automatically. After following this tutorial, your application is ready to go and supports tools to push code updates.