The development of an application does not end with the programming itself. After its completion, you need to upload it to the server so as the users can use it. This can be done in many ways. Manually or automatically. In this post, I will show you how to upload the application and new functionalities automatically to the server. I will use Gitlab CI/CD for this process, which will run unit tests and then upload the code to the hosting using the Deployer.
Why Deployer?
Deployer is a simple tool which allows you to deploy the application to the server. It automates the entire process which we would have to do manually without it. Nobody wants to do repetitive activities manually. Thanks to Deployer, we can create our own tasks and run them in the selected order. The tool already has built-in presets for various frameworks such as Symfony, Laravel, WordPress and many more. The undoubted advantage of Deployer is that it allows you to rollback the code to the previous version in case something goes wrong. It also has the ability to configure many deployment environments, so that before releasing the code to production, we can, for example, upload it to the test environment, so that the QA team can test the new functionality.
This tool allows you to deploy code in two ways. The first is to perform the entire build process on the server on which the application will be running. Deployer will then run commands on the target server where it will download the repository, dependencies, etc. Another way is to build the application once (locally or in the CI/CD) and upload the finished application to a selected place on the server. In this post I will show you both approaches.
Hosting Preparation
In order to upload an application to our hosting, the Deployer must have access to it. Connection to the hosting will be via SSH with key authorization. To enable such authorization on my hosting, it is required to generate a pair of keys using the ssh-keygen command, and then add the public key to the ~/.ssh/authorized_keys file. We will pass the private key to Deployer so that it can connect to the hosting.
Additionally, as mentioned above, Deployer will download the code from the repository on our hosting. Therefore, you need to provide access to the repository and other dependencies on your hosting. To do this, put the appropriate private keys into the ~/.ssh directory. This step is required only if we want to build the application on the target server.
Finally, we need to grant the appropriate permissions to the created files, e.g .:
- chmod 700 ~/.ssh
- chmod 600 ~/.ssh/*
Deployer Configuration
Installation
To install Deployer, run the command composer require deployer/deployer --dev which will add it to the vendor directory.
Then create the deploy.php file in the root directory of the application. You can do it manually or by executing the following command:
vendor/bin/dep init.
Configuration
As I mentioned before, Deployer offers built-in presets for deploying applications. In this case, I used the Symfony 4 preset. It contains, among other things, such part:
desc('Deploy project'); task('deploy', [ 'deploy:info', 'deploy:prepare', 'deploy:lock', 'deploy:release', 'deploy:update_code', 'deploy:shared', 'deploy:vendors', 'deploy:writable', 'deploy:cache:clear', 'deploy:cache:warmup', 'deploy:symlink', 'deploy:unlock', 'cleanup', ]);Here we can see the steps which will be performed after running the vendor/bin/dep deploy command. They are already implemented, but nothing prevents you from overwriting them or adding your own. You can even override the entire deploy command with a different list of steps in the argument.
Below, I paste the deploy.php file configured for my application. In the comments, I described what the individual elements are for.
<?php namespace Deployer; // Usage of built-in recipe require 'recipe/symfony4.php'; // Here we set the name of the directory in which the particular releases will be located set('application', 'myapp'); // Repository address from which the application will be downloaded set('repository', 'git@gitlab.com:xyz/app.git'); // Set the options with which the `composer install` command should be invoked set('composer_options', '{{composer_action}} --prefer-dist --no-progress --no-interaction --no-dev --optimize-autoloader'); // Set how many releases should be kept by Deployer on the server. // 3 means that we can go 3 releases back. -1 keeps all releases set('keep_releases', 3); // Specify which files are to be shared between the releases. // In my case, it will only be a file with environment variables add('shared_files', ['.env']); // List which directories are to be shared between releases. // In my case, it will only be a log directory add('shared_dirs', ['var/log']); // List of directories which must have write permission on the web server. add('writable_dirs', ['var/log', 'var/cache']); set('writable_mode', 'chown'); // we can choose from: chmod, chown, chgrp or acl. // Web server user set('http_user', 'xyz'); // Default stage. If no parameter is specified after calling the `dep deploy` command, // the code will be deployed into the stage defined here set('default_stage', 'prod'); set('ssh_multiplexing', true); // Configure the server to which the code will be deployed. // Provide here the parameters related to access, i.e. address, user or key path. // Additionally, we choose which git branch will be deployed and provide the directory where the application will appear. // We can define multiple such hosts in this file, e.g. additional one as a test environment host(getenv('HOSTING_HOST')) ->stage('prod') ->user(getenv('HOSTING_USER')) ->port(getenv('HOSTING_PORT')) ->identityFile(getenv('HOSTING_SSH_KEY_PATH')) ->addSshOption('StrictHostKeyChecking', 'no') ->set('branch', 'master') ->set('deploy_path', '/home/xyz/domains/xyz.pl/{{application}}') ->forwardAgent() ; // Set the path to the PHP version used by our application set('bin/php', function () { // return locateBinaryPath('php7.4'); return '/usr/local/bin/php74'; }); // I have overwritten the database migration command from the preset task('database:migrate', function () { $options = '{{console_options}} --allow-no-migration --all-or-nothing --no-interaction'; run(sprintf('{{bin/php}} {{bin/console}} doctrine:migrations:migrate %s', $options)); })->desc('Migrate database'); // In my case, to make the application available under the selected domain, the index.php file should be placed in the directory `/home/xyz/domains/xyz.pl/public_html`. // The current version can be found in `/home/xyz/domains/xyz.pl/myapp/current/` // Therefore, after a successful deployment, I create a symlink to the files in the deployment's public directory task('deploy:symlink', function () { run( "ln -nfs {{deploy_path}}/current/public/* {{deploy_path}}/../public_html" ); }); // After the cache is warmed up, I perform a database migration after('deploy:cache:warmup', 'database:migrate'); after('deploy:failed', 'deploy:unlock');The above configuration is enough to upload the application to the server manually from the command line. The only thing to do is to call the vendor/bin/dep deploy command, and the application will be available to users after a while.
Failure handling
Sometimes there may be a situation where changes uploaded to the server cause an error, and the application is not working properly.
We can then revert code to the previous version using the command:
The Deployer will then point the symbolic link back to the previous version, and the application will start running again on the previous code.
Gitlab CI/CD Configuration
In modern applications, the whole process of deploying is usually automated. Before the release of the new version we usually want to run static analysis of code or tests. Or build JavaScript and CSS files. It can take some time. So let's go a step further and use the Gitlab environment so that the entire process is done for us.
Implementation Scheme
In order for Gitlab to run its CI/CD environment, the .gitlab-ci.yml file should be added to the application. It contains the steps we want to run to deploy the code. In this case, we will divide the process into three stages:
- building an application with dev dependencies - PhpUnit and Deployer are added to composer in the require-dev section.
- on the application built this way we will run tests.
- if everything goes well, we will run the Deployer script, which will upload the code to the server and build it there.
As you can see in the configuration below, the first two steps will be performed on all branches of our repository. The third one will be launched only for the master branch, i.e. if you merge changes to this branch.
Environment Variables
Some environment variables are set in the .gitlab-ci.yml file, e.g. DOCKER_IMAGE_PHP. Others are not, such as PRIVATE_KEY.
We do not put confidential information in this file, because everyone would have access to it. We add them in Gitlab in the section Settings -> CI/CD -> Variables. There they will be available only to granted users.
Docker Image
The Docker image is used in each step. This is where all the code will be run — download code from the repository, download dependencies, test and deploy. It must therefore contain the programs necessary for these activities. In this case, the image looks like this:
FROM php:7.4-fpm-buster RUN apt-get -y update && apt-get -y --no-install-recommends install \ git \ zip \ unzip \ wget \ openssl \ curl \ openssh-client \ rsync COPY --from=composer /usr/bin/composer /usr/bin/composerYou need to build such image before and put it in the Gitlab container registry. This can be done by following the three steps below:
docker login registry.gitlab.com docker build -t registry.gitlab.com/xyz/app/php7.4 . docker push registry.gitlab.com/xyz/app/php7.4Building Application Within Gitlab CI/CD
There are cases when we might not want to build our application on the end server and do it on the Gitlab runner instead. This may be:
- we have multiple servers - then building application once is faster than doing it on each server
- we want to be sure that on each server the application will look exactly the same
- we cannot build assets because of lack of necessary tools like npm or yarn on the target server
- we want to keep the credentials in one place
The Deployer and Gitlab allows us to build application in such way. We only have to add small changes to the previous code.
Adjusting Deployer
To allow the Deployer to build the application in the Gitlab CI/CD, we need to change a bit our deploy.php file.
First of all, we have to add build task. It will tell Deployer to prepare the code in .build directory - it will download the repository and install dependencies.
It will also archive all files because sending one file is usually faster than sending thousands smaller. All those things will be done on Gitlab runner.
Secondly, in the upload task we have to tell Deployer where should it upload the prepared release. It will also unarchive the code on the target server.
task('build', function () { set('deploy_path', __DIR__ . '/.build'); invoke('deploy:prepare'); invoke('deploy:release'); invoke('deploy:update_code'); invoke('deploy:vendors'); cd('{{deploy_path}}/releases/1/'); run('rm -rfd deploy.php tests docker'); cd('{{deploy_path}}'); run("tar -cvf release.tar.gz -C {{deploy_path}}/releases/1/ $(find {{deploy_path}}/releases/1/ -maxdepth 1 -printf '%P ')"); })->local(); task('upload', function () { upload(__DIR__ . "/.build/release.tar.gz", '{{release_path}}'); cd('{{release_path}}'); run('tar -xf release.tar.gz'); run('rm release.tar.gz'); }); task('release', [ 'deploy:info', 'deploy:prepare', 'deploy:release', 'upload', 'deploy:shared', 'deploy:writable', 'deploy:cache:clear', 'deploy:cache:warmup', 'deploy:symlink', ]); task('deploy', [ 'build', 'release', 'cleanup', 'success' ]);Adjusting Gitlab Configuration File
In the .gitlab.yml file, the changes will be rather cosmetic. The only thing we have to add there is a private key to our repository, so as the runner has the access to it.
The deploy_prod part will now look like this:
Deploying Changes
Now, after merging the changes to the master branch, the runner will be launched and the application will be automatically deployed to our hosting. The entire process can be viewed in Gitlab in the CI/CD -> Pipelines tab. It looks like this:
On our hosting, in the directory /home/xyz/domains/xyz.pl/myapp we can see particular releases:
Deployer pushes changes to the releases/[release number] directory. As we set it in the deploy.php file, there will be three last versions here.
The current directory is a symbolic link to the last release.
In the shared directory there are files which are shared between individual releases.
Summary
Deployer is a simple tool which enables fast automatic deployment of applications. It is easy to configure and contains built-in settings we can use. In connection with Gitlab, it allows to automate the entire process of uploading the application, which saves a lot of time.
