How To Deploy Multiple PHP Applications using Ansible on Ubuntu 14.04

Introduction

This tutorial is the third in a series about deploying PHP applications using Ansible on Ubuntu 14.04. The first tutorial covers the basic steps for deploying an application; the second tutorial covers more advanced topics such as databases, queue daemons, and task schedulers (crons).

In this tutorial, we will build on what we learned in the previous tutorials by transforming our single-application Ansible playbook into a playbook that supports deploying multiple PHP applications on one or multiple servers. This is the final piece of the puzzle when it comes to using Ansible to deploy your applications with minimal effort.

We will be using a couple of simple Lumen applications as part of our examples. However, these instructions can be easily modified to support other frameworks and applications if you already have your own. It is recommended that you use the example applications until you are comfortable making changes to the playbook.

Prerequisites

To follow this tutorial, you will need:

• Two Droplets set up by following the first and second tutorials in this series.

• A new (third) Ubuntu 14.04 Droplet set up like the original PHP Droplet in the first tutorial, with a sudo non-root user and SSH keys. This Droplet which will be used to show how to deploy multiple applications to multiple servers using one Ansible playbook. We’ll refer to the IP addresses of the original PHP Droplet and this new PHP Droplet as your_first_server_ip and your_second_server_ip respectively.

• An updated /etc/hosts file on your local computer with the following lines added. You can learn more about this file in step 6 of this tutorial.

your_first_server_ip laravel.example.com one.example.com two.example.com
your_second_server_ip laravel.example2.com two.example2.com

The example websites we’ll use in this tutorial are laravel.example.com, one.example.com, and two.example.com. If you want to use your own domain, you’ll need to update your active DNS records instead.

Step 1 — Setting Playbook Variables

In this step, we will set up playbook variables to define our new applications.

In the previous tutorials, we hard-coded all of the configuration specifics, which is normal for many playbooks that perform specific tasks for a specific application. However, when you wish to support multiple applications or broaden the scope of your playbook, it no longer makes sense to hard code everything.

As we have seen before, Ansible provides variables which you can use in both your task definitions and file templates. What we haven’t seen yet is how to manually set variables. In the top of your playbook, alongside the hosts and tasks parameters, you can define a vars parameter, and set your variables there.

If you haven’t done so already, change directories into ansible-php from the previous tutorials.

cd ~/ansible-php/

Open up our existing playbook for editing.

nano php.yml

The top of the file should look like this:

---
- hosts: php
  sudo: yes

  tasks:
. . .

To define variables, we can just add in a vars section in, alongside hosts, sudo, and tasks. To keep things simple, we will start with a very basic variable for the www-data user name, like so:

---
- hosts: php
  sudo: yes

  vars:
    wwwuser: www-data

  tasks:
. . .

Next, go through and update all occurrences of the www-data user with the new variable {{ wwwuser }}. This format should be familiar, as we have used it within looks and for lookups.

To find and replace using nano, press CTRL+\. You’ll see a prompt which says Search (to replace):. Type www-data , then press ENTER. The prompt will change to Replace with:. Here, type {{ wwwuser }} and press ENTER again. Nano will take you through each instance of www-data and ask Replace this instanace?. You can press y to replace each one by one, or a to replace all.

Note: Make sure the variable declaration that we just added at the top isn’t changed too. There should be 11 instances of www-data that need to be replaced.

Before we go any further, there is something we need to be careful of when it comes to variables. Normally we can just add them in like this, when they are within a longer line:

- name: create /var/www/ directory
  file: dest=/var/www/ state=directory owner={{ wwwuser }} group={{ wwwuser }} mode=0700

However, if the variable is the only value in the string, we need to wrap it in quotes so the YAML parser can correctly understand it:

- name: Run artisan migrate
  shell: php /var/www/laravel/artisan migrate --force
  sudo: yes
  sudo_user: "{{ wwwuser }}"
  when: dbpwd.changed

In your playbook, this needs to happen any time you have sudo_user: {{ wwwuser }}. You can use a global find and replace the same way, replacing sudo_user: {{ wwwuser }} with sudo_user: “{{ wwwuser }}”. There should be four lines that need this change.

Once you have changed all occurrences, save and run the playbook:

ansible-playbook php.yml --ask-sudo-pass

There should be no changed tasks, which means that our wwwuser variable is working correctly.

Step 2 — Defining Nested Variables for Complex Configuration

In this section, we will look at nesting variables for complex configuration options.

In the previous step, we set up a basic variable. However, it is also possible to nest variables and define lists of variables. This provides the functionality we need to define the list of sites we wish to set up on our server.

First, let us consider the existing git repository that we have set up in our playbook:

- name: Clone git repository
  git: >
    dest=/var/www/laravel
    repo=https://github.com/do-community/do-ansible-adv-php.git
    update=yes
    version=example

We can extract the following useful pieces of information: name (directory), repository, branch, and domain. Because we are setting up multiple applications, we will also need a domain name for it to respond to. Here, we’ll use laravel.example.com, but if you have your own domain, you can substitute it.

This results in the following four variables that we can define for this application:

name: laravel
repository: https://github.com/do-community/do-ansible-adv-php.git
branch: example
domain: laravel.example.com

Now, open up your playbook for editing:

nano php.yml

In the top vars section, we can add in our application into a new application list:

---
- hosts: php
  sudo: yes

  vars:
    wwwuser: www-data

    applications:
      - name: laravel
        domain: laravel.example.com
        repository: https://github.com/do-community/do-ansible-adv-php.git
        branch: example

...

If you run your playbook now (using ansible-playbook php.yml --ask-sudo-pass), nothing will change because we haven’t yet set up our tasks to use our new applications variable yet. However, if you go to http://laravel.example.com/ in your browser, it should show our original application.

Step 3 — Looping Variables in Tasks

In this section we will learn how to loop through variable lists in tasks.

As mentioned previously, variable lists need looped over in each task that we wish to use them in. As we saw with the install packages task, we need to define a loop of items, and then apply the task for each item in the list.

Open up your playbook for editing:

nano php.yml

We will start with some easy tasks first. Around the middle of your playbook, you should find these two env tasks:

- name: set APP_DEBUG=false
  lineinfile: dest=/var/www/laravel/.env regexp='^APP_DEBUG=' line=APP_DEBUG=false

- name: set APP_ENV=production
  lineinfile: dest=/var/www/laravel/.env regexp='^APP_ENV=' line=APP_ENV=production

You will notice that they are currently hard-coded with the laravel directory. We want to update it to use the name property for each application. To do this we add in the with_items option to loop over our applications list. Within the task itself, we will swap out the laravel reference for the variable {{ item.name }}, which should be familiar from the formats we’ve used before.

It should look like this:

- name: set APP_DEBUG=false
  lineinfile: dest=/var/www/{{ item.name }}/.env regexp='^APP_DEBUG=' line=APP_DEBUG=false
  with_items: applications

- name: set APP_ENV=production
  lineinfile: dest=/var/www/{{ item.name }}/.env regexp='^APP_ENV=' line=APP_ENV=production
  with_items: applications

Next, move down to the two Laravel artisan cron tasks. They can be updated exactly the same as we just did with the env tasks. We will also add in the item.name into the name parameter for the cron entries, as Ansible uses this field to uniquely identify each cron entry. If we left them as-is, we would not be able to have multiple sites on the same server as they would overwrite each over constantly and only the last one would be saved.

The tasks should look like this:

- name: Laravel Scheduler
  cron: >
    job="run-one php /var/www/{{ item.name }}/artisan schedule:run 1>> /dev/null 2>&1"
    state=present
    user={{ wwwuser }}
    name="{{ item.name }} php artisan schedule:run"
  with_items: applications

- name: Laravel Queue Worker
  cron: >
    job="run-one php /var/www/{{ item.name }}/artisan queue:work --daemon --sleep=30 --delay=60 --tries=3 1>> /dev/null 2>&1"
    state=present
    user={{ wwwuser }}
    name="{{ item.name }} Laravel Queue Worker"
  with_items: applications

If you save and run the playbook now (using ansible-playbook php.yml --ask-sudo-pass), you should only see the two updated cron tasks as updated. This is due to the change in the name parameter. Apart from that, there have been no changes, and this means that our applications list is working as expected, and we have not yet made any changes to our server as a result of refactoring our playbook.

Step 4 — Applying Looped Variables in Templates

In this section we will cover how to use looped variables in templates.

Looping variables in templates is very easy. They can be used in exactly the same way that they are used in tasks, like all other variables. The complexity comes in when you consider file paths as well as variables, as in some uses we need to factor in the file name and even run other commands because of the new file.

In the case of Nginx, we need to create a new configuration file for each application, and tell Nginx that it should be enabled. We also want to remove our original /etc/nginx/sites-available/default configuration file in the process.

First, open up your playbook for editing:

nano php.yml

Find the Configure Nginx task (near the middle of the playbook), and update it as we have done with the other tasks:

- name: Configure nginx
  template: src=nginx.conf dest=/etc/nginx/sites-available/{{ item.name }}
  with_items: applications
  notify:
    - restart php5-fpm
    - restart nginx

While we are here, we will also add in two more tasks that were mentioned above. First, we will tell Nginx about our new site configuration file. This is done with a symlink between the sites-available and sites-enabled directories in /var/nginx/.

Add this task after the Configure nginx task:

- name: Configure nginx symlink
  file: src=/etc/nginx/sites-available/{{ item.name }} dest=/etc/nginx/sites-enabled/{{ item.name }} state=link
  with_items: applications
  notify:
    - restart php5-fpm
    - restart nginx

Next, we want to remove the default enabled site configuration file so it doesn’t cause problems with our new site configuration files. This is done easily with the file module:

- name: Remove default nginx site
  file: path=/etc/nginx/sites-enabled/default state=absent
  notify:
    - restart php5-fpm
    - restart nginx

Note that we didn’t need to loop applications, as we were looking for a single file.

The Nginx block in your playbook should now look like this:

- name: Configure nginx
  template: src=nginx.conf dest=/etc/nginx/sites-available/{{ item.name }}
  with_items: applications
  notify:
    - restart php5-fpm
    - restart nginx

- name: Configure nginx symlink
  file: src=/etc/nginx/sites-available/{{ item.name }} dest=/etc/nginx/sites-enabled/{{ item.name }} state=link
  with_items: applications
  notify:
    - restart php5-fpm
    - restart nginx

- name: Remove default nginx site
  file: path=/etc/nginx/sites-enabled/default state=absent
  notify:
    - restart php5-fpm
    - restart nginx

Save your playbook and open the nginx.conf file for editing:

nano nginx.conf

Update the configuration file so it uses our variables:

server {
    listen 80 default_server;
    listen [::]:80 default_server ipv6only=on;

    root /var/www/{{ item.name }}/public;
    index index.php index.html index.htm;

    server_name {{ item.domain }};

    location / {
        try_files $uri $uri/ =404;
    }

    error_page 404 /404.html;
    error_page 500 502 503 504 /50x.html;
    location = /50x.html {
        root /var/www/{{ item.name }}/public;
    }

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass unix:/var/run/php5-fpm.sock;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }
}

However, we haven’t finished yet. Notice the default_server at the top? We want to only include that for the laravel application, to make it the default. To do this we can use a basic IF statement to check if item.name is equal to laravel, and if so, display default_server.

It will look like this:

server {
    listen 80{% if item.name == "laravel" %} default_server{% endif %};
    listen [::]:80{% if item.name == "laravel" %} default_server ipv6only=on{% endif %};

Update your nginx.conf accordingly and save it.

Now it is time to run our playbook:

ansible-playbook php.yml --ask-sudo-pass

You should notice the Nginx tasks have been marked as changed. When it finishes running, refresh the site in your browser and it should be displaying the same as it did at the end of the last tutorial:

Queue: YES
Cron: YES

Step 5 — Looping Multiple Variables Together

In this step we will loop multiple variables together in tasks.

Now it is time to tackle a more complex loop example, specifically registered variables. In order to support different states and prevent tasks from running needlessly, you will remember that we used register: cloned in our Clone git repository task to register the variable cloned with the state of the task. We then used when: cloned|changed in the following tasks to trigger tasks conditionally. Now we need to update these references to support the applications loop.

First, open up your playbook for editing:

nano php.yml

Look down for the Clone git repository task:

- name: Clone git repository
  git: >
    dest=/var/www/laravel
    repo=https://github.com/do-community/do-ansible-adv-php.git
    update=yes
    version=example
  sudo: yes
  sudo_user: "{{ wwwuser }}"
  register: cloned

As we’re registering the variable in this task, we don’t need to do anything that we haven’t already done:

- name: Clone git repository
  git: >
    dest=/var/www/{{ item.name }}
    repo={{ item.repository }}
    update=yes
    version={{ item.branch }}
  sudo: yes
  sudo_user: "{{ wwwuser }}"
  with_items: applications
  register: cloned

Now, move down your playbook until you find the composer create-project task:

- name: composer create-project
  composer: command=create-project working_dir=/var/www/laravel optimize_autoloader=no
  sudo: yes
  sudo_user: "{{ wwwuser }}"
  when: cloned|changed

Now we need to update it to loop through both applications and cloned. This is done using the with_together option, and passing in both applications and cloned. As with_together loops through two variables, accessing items is done with item.#, where # is the index of the variable as it is defined. So for example:

with_together:
- list_one
- list_two

item.0 will refer to list_one, and item.1 will refer to list_two.

Which means that for applications we can access the properties via: item.0.name. For cloned we need to pass in the results from the tasks, which can be accessed via cloned.results, and then we can check if it was changed via item.1.changed.

This means the task becomes:

- name: composer create-project
  composer: command=create-project working_dir=/var/www/{{ item.0.name }} optimize_autoloader=no
  sudo: yes
  sudo_user: "{{ wwwuser }}"
  when: item.1.changed
  with_together:
    - applications
    - cloned.results

Now save and run your playbook:

ansible-playbook php.yml --ask-sudo-pass

There should be no changes from this run. However, we now have a registered variable working nicely within a loop.

Step 6 — Complex Registered Variables and Loops

In this section we will learn about more complicated registered variables and loops.

The most complicated part of the conversion is handling the registered variable we are using for password generation for our MySQL database. That said, there isn’t much more that we have to do in this step that we haven’t covered, we just need to update a number of tasks at once.

Open your playbook for editing:

nano php.yml

Find the MySQL tasks, and in our initial pass we will just add in the basic variables like we have done in previous tasks:

- name: Create MySQL DB
  mysql_db: name={{ item.name }} state=present
  with_items: applications

- name: Generate DB password
  shell: makepasswd --chars=32
  args:
    creates: /var/www/{{ item.name }}/.dbpw
  with_items: applications
  register: dbpwd

- name: Create MySQL User
  mysql_user: name={{ item.name }} password={{ dbpwd.stdout }} priv={{ item.name }}.*:ALL state=present
  when: dbpwd.changed

- name: set DB_DATABASE
  lineinfile: dest=/var/www/{{ item.name }}/.env regexp='^DB_DATABASE=' line=DB_DATABASE={{ item.name }}
  with_items: applications

- name: set DB_USERNAME
  lineinfile: dest=/var/www/{{ item.name }}/.env regexp='^DB_USERNAME=' line=DB_USERNAME={{ item.name }}
  with_items: applications

- name: set DB_PASSWORD
  lineinfile: dest=/var/www/{{ item.name }}/.env regexp='^DB_PASSWORD=' line=DB_PASSWORD={{ dbpwd.stdout }}
  when: dbpwd.changed

- name: Save dbpw file
  lineinfile: dest=/var/www/{{ item.name }}/.dbpw line="{{ dbpwd.stdout }}" create=yes state=present
  sudo: yes
  sudo_user: "{{ wwwuser }}"
  when: dbpwd.changed

- name: Run artisan migrate
  shell: php /var/www/{{ item.name }}/artisan migrate --force
  sudo: yes
  sudo_user: "{{ wwwuser }}"
  when: dbpwd.changed

Next we will add in with_together so we can use our database password. For our password generation, we need to loop over dbpwd.results, and will be able to access the password from item.1.stdout, since applications will be accessed via item.0.

We can update our playbook accordingly:

- name: Create MySQL DB
  mysql_db: name={{ item.name }} state=present
  with_items: applications

- name: Generate DB password
  shell: makepasswd --chars=32
  args:
    creates: /var/www/{{ item.name }}/.dbpw
  with_items: applications
  register: dbpwd

- name: Create MySQL User
  mysql_user: name={{ item.0.name }} password={{ item.1.stdout }} priv={{ item.0.name }}.*:ALL state=present
  when: item.1.changed
  with_together:
  - applications
  - dbpwd.results

- name: set DB_DATABASE
  lineinfile: dest=/var/www/{{ item.name }}/.env regexp='^DB_DATABASE=' line=DB_DATABASE={{ item.name }}
  with_items: applications

- name: set DB_USERNAME
  lineinfile: dest=/var/www/{{ item.name }}/.env regexp='^DB_USERNAME=' line=DB_USERNAME={{ item.name }}
  with_items: applications

- name: set DB_PASSWORD
  lineinfile: dest=/var/www/{{ item.0.name }}/.env regexp='^DB_PASSWORD=' line=DB_PASSWORD={{ item.1.stdout }}
  when: item.1.changed
  with_together:
  - applications
  - dbpwd.results

- name: Save dbpw file
  lineinfile: dest=/var/www/{{ item.0.name }}/.dbpw line="{{ item.1.stdout }}" create=yes state=present
  sudo: yes
  sudo_user: "{{ wwwuser }}"
  when: item.1.changed
  with_together:
  - applications
  - dbpwd.results

- name: Run artisan migrate
  shell: php /var/www/{{ item.0.name }}/artisan migrate --force
  sudo: yes
  sudo_user: "{{ wwwuser }}"
  when: item.1.changed
  with_together:
  - applications
  - dbpwd.results

Once you have updated your playbook, save it and run it:

ansible-playbook php.yml --ask-sudo-pass

Despite all of the changes we’ve made to our playbook, there should be no changes to the database tasks. With the changes in this step, we should have finished our conversion from a single application playbook to a multiple application playbook.

Step 7 — Adding More Applications

In this step we will configure two more applications in our playbook.

Now that we have refactored our playbook to use variables to define the applications, the process for adding new applications to our server is very easy. Simply add them to the applications variable list. This is where the power of Ansible variables will really shine.

Open your playbook for editing:

nano php.yml

At the top, in the vars section, find the applications block:

applications:
  - name: laravel
    domain: laravel.example.com
    repository: https://github.com/do-community/do-ansible-adv-php.git
    branch: example

Now add in two more applications:

applications:
  - name: laravel
    domain: laravel.example.com
    repository: https://github.com/do-community/do-ansible-adv-php.git
    branch: example

  - name: one
    domain: one.example.com
    repository: https://github.com/do-community/do-ansible-php-example-one.git
    
    branch: master

  - name: two
    domain: two.example.com
    repository: https://github.com/do-community/do-ansible-php-example-two.git
    branch: master

Save your playbook.

Now it is time to run your playbook:

ansible-playbook php.yml --ask-sudo-pass

This step may take a while as composer sets up the new applications. When it has finished, you will notice a number of tasks are changed, and if you look carefully you’ll notice that each of the looped items will be listed. The first, our original application should say ok or skipped, while the new two applications should say changed.

More importantly, if you visit all three of the domains for your configured sites in your web browser you should notice three different websites.

The first one should look familiar. The other two should display:

This is example app one!

and

This is example app two!

With that, we have just deployed two new web applications by simply updating our applications list.

Step 8 — Using Host Variables

In this step we will extract our variables to host variables.

Taking a step back, playbook variables are good, but what if we want to deploy different applications onto different servers using the same playbook? We could do conditional checks on each task to work out which server is running the task, or we can use host variables. Host variables are just what they sound like: variables that apply to a specific host, rather than all hosts across a playbook.

Host variables can be defined inline, within the hosts file, like we’ve done with the ansible_ssh_user variable, or they can be defined in dedicated file for each host within the host_vars directory.

First, create a new directory alongside our hosts file and our playbook. Call the directory host_vars:

mkdir host_vars

Next we need to create a file for our host. The convention Ansible uses is for the filename to match the host name in the hosts file. So, for example, if your hosts file looks like this:

[php]
your_first_server_ip ansible_ssh_user=sammy

Then you should create a file called host_vars/your_first_server_ip. Let’s create that now:

nano host_vars/your_first_server_ip

Like our playbooks, host files use YAML for their formatting. This means we can copy our applications list into our new host file, so it looks like this:

---
applications:
  - name: laravel
    domain: laravel.example.com
    repository: https://github.com/do-community/do-ansible-adv-php.git
    branch: example

  - name: one
    domain: one.example.com
    repository: https://github.com/do-community/do-ansible-php-example-one.git
    branch: master

  - name: two
    domain: two.example.com
    repository: https://github.com/do-community/do-ansible-php-example-two.git
    branch: master

Save your new hosts file, and open your playbook for editing:

nano php.yml

Update the top to remove the entire applications section:

---
- hosts: php
  sudo: yes

  vars:
    wwwuser: www-data

  tasks:
. . .

Save the playbook, and run it:

ansible-playbook php.yml --ask-sudo-pass

Even though we have moved our variables from our playbook to our host file, the output should look exactly the same, and there should be no changes reported by Ansible. As you can see, host_vars work in the exact same way that vars in playbooks do; they are just specific to the host.

Variables defined in host_vars files will also be accessible across all playbooks that manage the server, which is useful for common options and settings. However, be careful not to use a common name that might mean different things across different playbooks.

Step 9 — Deploying Applications on Another Server

In this step we will utilize our new host files and deploy applications on a second server.

First, we need to update our hosts file with our new host. Open it for editing:

nano hosts

And add in your new host:

[php]
your_first_server_ip ansible_ssh_user=sammy
your_second_server_ip ansible_ssh_user=sammy

Save and close the file.

Next, we need to create a new hosts file, like we did with the first.

nano host_vars/your_second_server_ip

You can pick one or more of our example applications and add them into your host file. For example, if you wanted to deploy our original example and example two to the new server, you could use:

---
applications:
  - name: laravel
    domain: laravel.example2.com
    repository: https://github.com/do-community/do-ansible-adv-php.git
    branch: example

  - name: two
    domain: two.example2.com
    repository: https://github.com/do-community/do-ansible-php-example-two.git
    branch: master

Save your playbook.

Finally we can run our playbook:

ansible-playbook php.yml --ask-sudo-pass

Ansible will take a while to run because it is setting everything up on your second server. When it has finished, open up your chosen applications in your browser (in the example, we used laravel.example2.com two.example2.com)and to confirm they have been set up correctly. You should see the specific applications that you picked for your host file, and your original server should have no changes.

Conclusion

This tutorial took a fully functioning single-application playbook and converted it to support multiple applications across multiple servers. Combined with the topics covered in the previous tutorials, you should have everything you need to write a full playbook for deploying your applications. As per the previous tutorials, we still have not logged directly into the servers using SSH.

You will have noticed how simple it was to add in more applications and another server, once we had the structure of the playbook worked out. This is the power of Ansible, and is what makes it so flexible and easy to use.