Submitted by douglas.w.needham on

One of the driving ideas in everything I do is the principle of "Don't Repeat Yourself" (DRY). While primarily a software development principle, I also apply it to tasks I perform. For example, when I did not have a working Cobbler and Koan instance able to handle the latest RHEL release for installing virtual machines, rather than typing out commands with over a dozen arguments each time, I edited a file to create a shell script to do the task once, and then each new machine copied that file, got tweaked for the new machine (e.g. a new hostname and MAC addresses), the same was done with a kickstart file, and then I would simply type a command such as `. newhost.sh` to install that new host. That way, I could with minimal duplication, effort and exposure repeat a proven process to install new machines. And if things were not quite right, like if I wanted to add more disk space, all I had to do is delete the existing VM, tweak one or both of the files for that host, lather, rinse and repeat.  Could I have done more? Sure! I could have parameterized things so that I was not copying the entire resulting command, but my goal was to make something repeatable and less error prone, so that I could eventually get Cobbler and Koan back up and running. It is also why I have become a big fan of things like Ansible, where I can create roles to reuse every time I build a new machine, and why I have been re-building a Jenkins server.

Where does Drupal come into play? Well, every few days, either the core of Drupal gets an update or some module does, and I like to stay fully up-to-date. And because I have both a development/test server and my production server, I again prefer to repeat myself as little as possible. And just how much repetition is there? Well, without Jenkins, tonight to apply the latest update, I would have had to do the following:

On the development server, I would start by doing:

  1. `composer update`      (To update to the latest packages)
  2. `drush cr`                     (cache rebuild)
  3. `drush updb`               (update database)
  4. `drush cr`                     (to again rebuild the cache)
  5. Testing, config changes, etc.
  6. `drush cex`                  (configuration export)
  7. `git commit`                (adding in the changed files, of course)
  8. `git push`                    (to get things into the main repository)

Then, on the production server, this becomes:

  1. `git pull`
  2. `composer install`       (To install the new packages)
  3. `drush cr`
  4. `drush cim`                  (configuration import)
  5. `drush updb`
  6. `drush cr`

As you can see, there is considerable repetition there, particularly when done a couple of times a week.

Jenkins pipelines to the rescue. Now, when I do that `git push`, within a few minutes, all the steps on the production server are taken care of. But it is far from perfect. First, all the steps on the development server are still manual; I may think about automating some of that later using a Jenkins pipeline input step and some rearrangement of the steps. But there are two other concerns of higher importance... the starting of deployment to the production server is not optimal, and there are possible issues with doing all the steps in production automatically.

On the first the options are somewhat limited. I could just login to Jenkins, find the pipeline, and click "Build"... but that is more repeating of my actions. Or, I could accept delays and either just build it periodically, or poll the git repository periodically. Right now, I am doing the latter, and polling every 15 minutes. But the optimum solution is to have the git repository notify my Jenkins instance when it receives a push. No polling and no delays, it just starts the build deployment. But that means getting the notification through my firewall and across my network to the Jenkins server, which is something I will do soon, but not immediately.

On the second, it is actually the bigger issue. In doing the `drush cim`, there is no human involved in double checking the deployment. So if between deployments, somebody does a configuration change, but does not capture that fact so that it gets fed back into development and picked up there, a deployment change could get undone and cause considerable grief. It is far less likely to happen in my personal environment, but if you have multiple developers, multiple development and testing layers, etc., it complicates things and can make it much more likely to happen.

As for the details, well, here were the steps.

  1. In Jenkins, create a new SSH credential to use to connect to the production machine.
  2. Create the script file, which you can see here.
  3. In Jenkins, create a new item and make it a pipeline.
    1. Fill in the description.
    2. Tell it not to allow concurrent builds.
    3. Tell it that it was a GitHub project, with the URL of the repository
    4. Tell it to poll the SCM, with a schedule of `H/15 * * * *`
    5. Tell it that the definition of the pipeline was from the SCM, with the parameters of:
      1. The SCM is Git
      2. The repository URL (the same)
      3. What credentials to use.
    6. What branch to build.
    7. The path for the script file.

Once all is said and done, every 15 minutes (at some random offset, since I use the 'H' instead of '*' on the first field of the schedule), it checks the revision, and if needed, checks out the repository, runs the script, and things get deployed to production.

Granted, this is not as involved as it could be. I could be deploying docker images, and may at some point do that, but until then...

Categories
Jenkins
Drupal