A BDD guide for Ansible

Many many years ago I introduced Test Driven Development to a team with a significant amount of legacy code. We removed over forty thousand lines of dead code in the first 3 months. Over time we were damn near 100% code coverage with a very lean code base. It was beautiful! However I found we had a lot of code testing different exceptions that were never going to be reached based on user actions. Also the linkage between our test cases and requirements were weak at best.

Later in my development life I discovered BDD which addressed this concern. It did so by making user stories a living document in my code which drove my test. This resulted in my team only writing code that the user story covers. It was very profound for coding however it was  missing from the deployment activities.

Deployments and by extension deployment verification tests (DVT) are an important part of the software development lifecycle. Especially now that we live in a DevOps and SRE world.  One of the many provisioning tools/frameworks we use is Ansible. This article is based on a recent enablement I did for a team to apply BDD principles to Ansible.

Step 1: Don’t develop ansible on it’s own

Ansible on it’s own doesn’t provide any unit test or linting capability. If you leverage “ansible-galaxy init <playbook name>” to create a role then you simply have a test playbook to leverage. Tools like ServerSpec help but are not enough. You need something like Molecule to bring together more software development best practices to ansible.

The basic workflow molecule provides is:

  • Verify the syntax of the role (ansible-lint)
  • Create the virtual images used for testing via a Vagrant wrapper. Alternatively it also supports Docker and OpenStack.
  • “Converge” will run ansible playbook to provision the image
  • Idempotence” – Run the role again in order to verify it doesn’t change anything.
  • Verify by linting your test and running them.

Let’s take this one by one. The integration of linting  into the deployment script development process is welcomed. Leveraging ansible-lint, flake8, and rubocop helps keep software somewhat to best practices when collaborating. While it is possible to accomplish all this without molecule it is nice to have something with these tools baked in

One of the big things to understand about Molecule is that it is YAML. Meaning it is very easy to understand and is widely used for configurations. So while you could just use Vagrant to provision a test server it requires your team to know Ruby. Actually, let me restate that. You don’t need to know all of ruby but you do need to know the syntax. Nevertheless unless you are creating a Ruby application YAML may be less effort for the entire team to adopt.

Idempotence” is an interesting one and frankly a hard sale for people. You only want your script to make changes that need to be changed. So if you run your script twice it should not change anything on the 2nd run because nothing needs to change. This sounds trivial but in practice is quite difficult. Once you leverage a “command” or “shell” task to perform external actions you now need to add logic to prevent it from being run twice.  Thus the hard sale but it enables you to run the script many times on the same server to preserve the configuration.

To install Molecule run

pip install molecule

At this point I setup my role project using

ansible-galaxy init ansible-role-bdd-example
cd ansible-role-bdd-example
molecule init

You now have a skeleton project but you are not quite ready to code.

Step 2: Go sandbox yourself

I was running a workshop on Ansible a few months ago and lost the first hour to dependency conflicts in Python modules. Personally I am not a fan of virtualenv. It is solving a problem of dependencies in an environment that should not exist. In my opinion other languages and package managers have better solutions. Nevertheless since that workshop I use it for anything Python based.

To install virtualenv:

pip install virtualenv

Now we need to create a new environment sandbox to work with.

virtualenv venv

The newly created “venv/” directory acts as a blank slate for python development.  to activate it run:

source venv/bin/activate

For Windows machines leave out the “source” part of the command. To escape your sandbox run “deactivate” from the command line. Now that we have our environment we need to populate it using a requirements file.  For the purpose of this tutorial all we need is:

molecule>=1.25.0
python-vagrant>=0.5.15
pytest-bdd>=2.18.2
testinfra>=1.5.5

Now to prevent the molecule linting to go crazy we need to ensure it stays out of your virtual environment directory by adding the following to your molecule.yml:

molecule:
 ignore_paths:
 - .git
 - .vagrant
 - .molecule
 - venv

Step 3: Know your use-cases

Often when I ask why a configuration is a specific way on a server the response is “it is a best practice.” Then once I dig into it I find it is a best practice for a use-case that isn’t ours or no longer exist. When leveraging BDD with living documents in your code you know what features are associated with a configuration. Remember those features are driving the configuration test.

There is plenty of methods for capturing use-cases. I always like leveraging the typical agile user story template

As a <role>, 
I want <goal/desire> 
So that <benefit>

So for sake of this article let’s say your web developers have a use-case that looks like:

 As a web developer
 I want a server running nginx
 So that it can host my web assets

Your security engineer on the other hand will have a different use-case for the same feature:

As a security engineer
I want the nginx server to be running the latest packages
So that all the security patches have been applied

I recommend grouping these use-cases together by feature and adding them to a Gherkin feature file. The basic syntax of a feature file is:

 Feature: Some terse yet descriptive text of what is desired
   Textual description of the business value of this feature
   Business rules that govern the scope of the feature
   Any additional information that will make the feature easier to understand
 
   Scenario: Some determinable business situation
     Given some precondition
       And some other precondition
     When some action by the actor
       And some other action
       And yet another action
     Then some testable outcome is achieved
       And something else we can check happens too

I normally use the feature description area to store my use-cases. Additionally you can use scenario outlines to leverage test data sets. Such as this example:

Feature: Web Hosting
 As a web developer
 I want a server running nginx
 So that it can host my web assets

 As a security engineer
 I want the nginx server to be running the latest packages
 So that all the security patches have been applied
 Scenario: Running a http server
   Given nginx is installed
   When the server is running
   Then port 80 is open

 Scenario Outline: Expect security updates to be installed
   Given the package <package>
   When the version is fetched
   Then It should be equal or later than version <version>

   Examples: Ubuntu snapshot
     | package      | version          |
     | nginx        | 1.4.6-1ubuntu3.7 |
     | nginx-common | 1.4.6-1ubuntu3.7 |
     | nginx-core   | 1.4.6-1ubuntu3.7 |

Step 4: Write your test

Out of the box molecule supports both TestInfra and ServerSpec. You can get flavors of cucumber to work with both of them. However the easiest way to integrate cucumber with molecule without using a separate runner is  TestInfra with pytest-bdd. The reason for this is both TestInfra and pytest-bdd are both extensions of  pytest. Solutions like behave and other cucumber implementations require a bit more duct tape and bubble gum to get the same integration. That being said you do need some bubble gum.

In order to make TestInfra work with molecule you need to generate a host object using the connections api. Since molecule generates an inventory on the fly you need to add the following:

import testinfra

host = testinfra.get_host(
 "ansible://all?ansible_inventory=.molecule/ansible_inventory",
 sudo=True)

Next you need to add example converters for each scenario which uses a scenario outline .

@scenario('../features/web_hosting.feature',
 'Expect security updates to be installed',
 example_converters=dict(package=str, version=str))
def test_package_scenario():
 '''
 scenarios with tables that require type mapping must be referenced
 directly before calling "scenarios()"
 '''
 pass

After that you can use “scenarios(‘../features’)” to pull in the remaining scenarios. Now you can simply write pytest-bdd code using the TestInfra host object.

@given(parsers.parse('{package:w} is installed'))
def nginx_is_installed(package):
 assert host.package(package).is_installed


@when('the server is running')
def the_server_is_running():
 assert host.service('nginx').is_running


@then(parsers.parse('port {port:d} is open'))
def port_80_is_open(port):
 uri = 'tcp://%d' % port
 host.socket(uri).is_listening

Step 5: Do what you would have normally done first.

Now that you have features documented and test written it is time to write your ansible code. Seems backwards but there is a very important reason why.  It is the best way to ensure your have 100% coverage of your use-cases without dead code. You will not have any dead code since you only implemented enough to make the test pass and no more. So in the end it is equal or less work than the other way around.

 

Leave a Reply