A BDD guide for Ansible

Many many years ago I introduced Test Driven Development to a team with a significant amount of legacy code. We removed over forty thousand lines of dead code in the first 3 months. Over time we were damn near 100% code coverage with a very lean code base. It was beautiful! However I found we had a lot of code testing different exceptions that were never going to be reached based on user actions. Also the linkage between our test cases and requirements were weak at best.

Later in my development life I discovered BDD which addressed this concern. It did so by making user stories a living document in my code which drove my test. This resulted in my team only writing code that the user story covers. It was very profound for coding however it was  missing from the deployment activities.

Deployments and by extension deployment verification tests (DVT) are an important part of the software development lifecycle. Especially now that we live in a DevOps and SRE world.  One of the many provisioning tools/frameworks we use is Ansible. This article is based on a recent enablement I did for a team to apply BDD principles to Ansible.

Step 1: Don’t develop ansible on it’s own

Ansible on it’s own doesn’t provide any unit test or linting capability. If you leverage “ansible-galaxy init <playbook name>” to create a role then you simply have a test playbook to leverage. Tools like ServerSpec help but are not enough. You need something like Molecule to bring together more software development best practices to ansible.

The basic workflow molecule provides is:

  • Verify the syntax of the role (ansible-lint)
  • Create the virtual images used for testing via a Vagrant wrapper. Alternatively it also supports Docker and OpenStack.
  • “Converge” will run ansible playbook to provision the image
  • Idempotence” – Run the role again in order to verify it doesn’t change anything.
  • Verify by linting your test and running them.

Let’s take this one by one. The integration of linting  into the deployment script development process is welcomed. Leveraging ansible-lint, flake8, and rubocop helps keep software somewhat to best practices when collaborating. While it is possible to accomplish all this without molecule it is nice to have something with these tools baked in

One of the big things to understand about Molecule is that it is YAML. Meaning it is very easy to understand and is widely used for configurations. So while you could just use Vagrant to provision a test server it requires your team to know Ruby. Actually, let me restate that. You don’t need to know all of ruby but you do need to know the syntax. Nevertheless unless you are creating a Ruby application YAML may be less effort for the entire team to adopt.

Idempotence” is an interesting one and frankly a hard sale for people. You only want your script to make changes that need to be changed. So if you run your script twice it should not change anything on the 2nd run because nothing needs to change. This sounds trivial but in practice is quite difficult. Once you leverage a “command” or “shell” task to perform external actions you now need to add logic to prevent it from being run twice.  Thus the hard sale but it enables you to run the script many times on the same server to preserve the configuration.

To install Molecule run

pip install molecule

At this point I setup my role project using

ansible-galaxy init ansible-role-bdd-example
cd ansible-role-bdd-example
molecule init

You now have a skeleton project but you are not quite ready to code.

Step 2: Go sandbox yourself

I was running a workshop on Ansible a few months ago and lost the first hour to dependency conflicts in Python modules. Personally I am not a fan of virtualenv. It is solving a problem of dependencies in an environment that should not exist. In my opinion other languages and package managers have better solutions. Nevertheless since that workshop I use it for anything Python based.

To install virtualenv:

pip install virtualenv

Now we need to create a new environment sandbox to work with.

virtualenv venv

The newly created “venv/” directory acts as a blank slate for python development.  to activate it run:

source venv/bin/activate

For Windows machines leave out the “source” part of the command. To escape your sandbox run “deactivate” from the command line. Now that we have our environment we need to populate it using a requirements file.  For the purpose of this tutorial all we need is:

molecule>=1.25.0
python-vagrant>=0.5.15
pytest-bdd>=2.18.2
testinfra>=1.5.5

Now to prevent the molecule linting to go crazy we need to ensure it stays out of your virtual environment directory by adding the following to your molecule.yml:

molecule:
 ignore_paths:
 - .git
 - .vagrant
 - .molecule
 - venv

Step 3: Know your use-cases

Often when I ask why a configuration is a specific way on a server the response is “it is a best practice.” Then once I dig into it I find it is a best practice for a use-case that isn’t ours or no longer exist. When leveraging BDD with living documents in your code you know what features are associated with a configuration. Remember those features are driving the configuration test.

There is plenty of methods for capturing use-cases. I always like leveraging the typical agile user story template

As a <role>, 
I want <goal/desire> 
So that <benefit>

So for sake of this article let’s say your web developers have a use-case that looks like:

 As a web developer
 I want a server running nginx
 So that it can host my web assets

Your security engineer on the other hand will have a different use-case for the same feature:

As a security engineer
I want the nginx server to be running the latest packages
So that all the security patches have been applied

I recommend grouping these use-cases together by feature and adding them to a Gherkin feature file. The basic syntax of a feature file is:

 Feature: Some terse yet descriptive text of what is desired
   Textual description of the business value of this feature
   Business rules that govern the scope of the feature
   Any additional information that will make the feature easier to understand
 
   Scenario: Some determinable business situation
     Given some precondition
       And some other precondition
     When some action by the actor
       And some other action
       And yet another action
     Then some testable outcome is achieved
       And something else we can check happens too

I normally use the feature description area to store my use-cases. Additionally you can use scenario outlines to leverage test data sets. Such as this example:

Feature: Web Hosting
 As a web developer
 I want a server running nginx
 So that it can host my web assets

 As a security engineer
 I want the nginx server to be running the latest packages
 So that all the security patches have been applied
 Scenario: Running a http server
   Given nginx is installed
   When the server is running
   Then port 80 is open

 Scenario Outline: Expect security updates to be installed
   Given the package <package>
   When the version is fetched
   Then It should be equal or later than version <version>

   Examples: Ubuntu snapshot
     | package      | version          |
     | nginx        | 1.4.6-1ubuntu3.7 |
     | nginx-common | 1.4.6-1ubuntu3.7 |
     | nginx-core   | 1.4.6-1ubuntu3.7 |

Step 4: Write your test

Out of the box molecule supports both TestInfra and ServerSpec. You can get flavors of cucumber to work with both of them. However the easiest way to integrate cucumber with molecule without using a separate runner is  TestInfra with pytest-bdd. The reason for this is both TestInfra and pytest-bdd are both extensions of  pytest. Solutions like behave and other cucumber implementations require a bit more duct tape and bubble gum to get the same integration. That being said you do need some bubble gum.

In order to make TestInfra work with molecule you need to generate a host object using the connections api. Since molecule generates an inventory on the fly you need to add the following:

import testinfra

host = testinfra.get_host(
 "ansible://all?ansible_inventory=.molecule/ansible_inventory",
 sudo=True)

Next you need to add example converters for each scenario which uses a scenario outline .

@scenario('../features/web_hosting.feature',
 'Expect security updates to be installed',
 example_converters=dict(package=str, version=str))
def test_package_scenario():
 '''
 scenarios with tables that require type mapping must be referenced
 directly before calling "scenarios()"
 '''
 pass

After that you can use “scenarios(‘../features’)” to pull in the remaining scenarios. Now you can simply write pytest-bdd code using the TestInfra host object.

@given(parsers.parse('{package:w} is installed'))
def nginx_is_installed(package):
 assert host.package(package).is_installed


@when('the server is running')
def the_server_is_running():
 assert host.service('nginx').is_running


@then(parsers.parse('port {port:d} is open'))
def port_80_is_open(port):
 uri = 'tcp://%d' % port
 host.socket(uri).is_listening

Step 5: Do what you would have normally done first.

Now that you have features documented and test written it is time to write your ansible code. Seems backwards but there is a very important reason why.  It is the best way to ensure your have 100% coverage of your use-cases without dead code. You will not have any dead code since you only implemented enough to make the test pass and no more. So in the end it is equal or less work than the other way around.

 

Introducing Logan Alec Cohen and a lesson in managing stakeholders

My wife and I are very analytical which means we did a lot of research prior to Logan’s birth. Maybe a bit too much. There is plenty of research and documentation available on how to have a child and keep it alive for the first month. However that child is not your child.

This is true in the same way not all pregnant women get the same food cravings. So one parent’s experience will result with a baby which will nurse with ease while another will prefer formula.  Of course navigating all this as a first time parent is next to impossible which is where subject matter experts come into the picture in the form of doctors, nurses, and specialist.  Unfortunately not all of them agree. I never thought keeping my son alive would resemble a software project. I ended up treating the all the medical experts as stakeholders in a project called “Logan Alec Cohen’s first few weeks of life.”

For instance, a pediatrician’s top priority is for the child to hit specific weight targets. So he will recommend introducing bottles, formula, and increased number of feedings. A lactation specialist will be concerned that the child will be harder to nurse with the introduction of the bottle and lack of sleep for the mom will result with less milk production. Of course to add to the confusion they both want to overrule the other.

On one hand it is easy to say one is a Doctor and one is a specialized nurse. So the Doctor’ recommendation has higher priority. However we also experienced two pediatricians from the same practice giving different advice. Where one pediatrician that responded to a call over the weekend wants to prioritize formula and our primary wants it used only as a supplement. If I put on a project manager hat I would setup a meeting between the disagreeing experts and build consensus. However due to their availability that is not possible. So simply prioritizing stakeholders is not always easy..

A typical method of prioritizing stakeholders is by role, influence, and interest/availability. That last bit is key. Availability gives you a point in time prioritization for evaluating feedback. The doctor on call during the weekend get prioritization because they are available at that time. A way we have learned to address volatility of opinions is to ask each expert to write down their opinion with justification. We then keep a folder with all the opinions and give it to the expert we are consulting at that time.

 

Embracing Behavior Driven Development

http://dilbert.com/strip/2006-02-26

Many years Ago I worked on a project which became Rational Team Concert 1.0. The ability (via OSLC) to link to all of the development assets made life easier. I could easily click from requirements to test results. Today I spend majority of my days in github which doesn’t have the same type of linkage. While linkage made my life easier it did not mean the assets were in sync which caused greater overhead. Recently I adopted BDD (Behavior Driven Development) and found myself using it for…everything.

Frankly it just makes sense to use it for everything from javascript applications to infrastructure ansible playbooks. All of your requirements in one place with your code and it encourages better requirements. It sounds too good to be true and unfortunately it can be a hard sell to others. Especially since the main advertised use case for BDD is to help the business owner/requirements author which don’t always have a strong presence on smaller projects.

I recall a few projects where I spent majority of my time calling myself an architect and converting business requirements to development requirements & test cases for development. Frankly it was like playing a game of telephone.  In software development the best way to ensure requirements are met are to have less middle men.

I have learned the hard way that documenting requirements is important. Even if you think it is for disposable code. On one hand it forces you to think about what you are going to write. So you spend less time rewriting your code. However on the other hand projects have a habit of lasting far longer than they should.  Your future self will thank you for documenting.

Better requirements

I started my IBM career in the Rational acquisition back in 2003. Home of requirements, governance, testing, and traceability software. I have an entire book on gathering and writing requirements that I quote from more often than I should. Nevertheless a good project manager, architect, designer, or anyone else in a requirements gathering role is not always available for projects. So a simple language/framework like Gherkin that anyone can use is far better than nothing.

While I was a Teaching Assistant for the introduction to computer science class at Clark University I taught students to outline preconditions and postconditions for each method before writing a line of code. Gherkin is essentially the same thing with given, when, and then. “Given” is your precondition, “when” is your method action, and “then” is your postcondition. You write them for each scenario of each feature.

Features

BDD documentation is different from other project related documentation. It isn’t a substitute for a decisions document or design thinking outputs. Those are all point in time documents. A BDD Feature is a living document which outlines the current expectation for a solution’s specific feature,

Think about how a typical development project is managed. You have an agile story or change request for the solution to implement. Then over time you have additional stories or change requests to change that behavior. An archeological dig through documents, development assets, and meeting notes are required to grasp current behavior.

The basic schema of a feature document is as follows:

Feature: <feature name>
    <Feature Description>

    Background:
        Given <precondition>
        And <precondition>

    Scenario: <scenario name>
        Given <precondition>
        Where <action>
        Then <postcondition

Now of course it can get far more complicate but that is the basic gist. It is human readable and can be used to describe the solution, component, or system role features.

The Glue

More documentation is all good but it isn’t code. Text only has impact if it can pass/fail code. That is where step code comes in. Now depending on which language you are using step code will look slightly different however it will look something like this:

@given(’text’)
function setup_scenario_x (test_context) {
    …
}

Each step is a method to match to feature document text, an action to perform, and a scoped variable to the test. Yes this is essentially a form of unit test at the end of the day but it provides very different insight.

End of the day

Up to recently I was a born again test driven developer. I would translate my requirements into an architecture decision document, then to component specifications, then to tests, and lastly write my code. This process over time proved less and less agile. Constant change made this inflexible. Majority of my test were written to ensure my code addressed null pointer exceptions and reach 100% coverage. While important what is critical for a minimum viable product is just enough code to meet the business requirements.

For more information about BDD and a great framework to get you started go to the cucumber project.

Back to basics

It seems that every time I get in a conversation with snugug he tells me to avoid leveraging frameworks Now I still stand by my belief that frameworks are inevitable however I thought I would give it a try with a small proof of concept. In fact I would try to use as few libraries as possible and try to  just use vanilla javascript.

Library freedom and curse

Normally I just use whatever libraries for development that the large framework suggest. So I use intern.io for testing Dojo, protractor for testing AngularJs, etc… On one hand this provides a immense amount of freedom and on the other hand adds significant overhead. Selecting a library is like selecting a restaurant for lunch next year from today’s yelp reviews. A thorough evaluation of the library capabilities, it’s community, and expected enhancements needs to be performed and alternatives considered. I can’t tell you how much time I lost comparing mocha to jasmine.

Even if you don’t leverage any libraries in your application and stick to standards you are faced with the very ugly truth. Not every browser implements standards the same way. Making up for this gap requires polyfills which results in the same overhead mentioned for selecting libraries.

Of course you could roll your own but frankly something as simple as XMLHttpRequest can be a nightmare. My favorite was finding out that in IE 9 the console object is undefined unless the developer tools are open. Don’t get me started about the hoops you need to jump though to get the PhantomJS Browser working.

Nothing more than NPM

Builds starts off simple and quickly gets very complicated. Gulp works especially well for complicated builds but it just ends up being more code to manage. The alternative is to just use NPM as a build tool. It works surprisingly well but I’m guessing there is an upper limit to how complicated your build can be since pre and post hooks can only get you so far. That being said I would suggest leveraging just NPM for build management until you actually need those additional capabilities. I should mention that I found it slower and sometimes wished I had used broccoli.

 Conclusion

Many years ago I was brought in to address a project that was drowning in technical debt. It was 60,000 lines of Perl code. The head developer at the time didn’t trust modules or 3rd party libraries. He wrote everything himself so he could optimize it. The result was 6 weeks to resolve a defect and the time to bring on new developers was 4 months.

My first order of business was to draw boxes around code, look for duplication with modules on CPAN, and replace them. The result was a more manageable 5,000 lines of code.  The interesting thing was performance got better. Mainly because even though the libraries were bigger they used new faster features of the language. The lesson I learned from that experience is you need to size your code for the resources maintaining it.

There is cost associated with using a framework, library, or micro-library. However there is also a cost with not using them. Shared code is always bloated but it gets updated more often with defect fixes and possibly faster techniques. I am not saying you should or should not use frameworks like Angular, React, Ember, etc… However you should understand your capabilities as a team and balance that against the end user experience.

This was a great experiment and as result I will only bring in frameworks as needed moving forward.

Framework fatigue

My personal belief is that coding frameworks are natural and can’t be avoided. Especially as projects mature and grow in size. They start as boilerplate code, best practices, and style guides. Then code is refactored into more manageable components and a framework emerges. That framework is used across many different projects and inherits use-cases that may not be relevant to your project and can be seen as bloat.  Bloat leads to performance implications which then leads to considering a different framework or writing your own.

Migrating to a different framework can be extensive and for front end developers happen more often than the backend. I loathe rewrites and find them to be an anti-pattern. However there are  justified reasons for making them.

There comes a time when technical debt is so great it justifies a major change. For frameworks this means it is easier to migrate to a new framework than address the technical debt you have with the existing framework. In my experience the technical debt that drives this is less performance and more security or maintainability.

Up to this year I would have never thought on-boarding new developers could drive a framework change. I was in the camp a basic Computer Science degree as a solid foundation was enough. However majority of mastering a framework is less learning the terminology/usage and more understanding the community.  Many of the frameworks today you just can’t buy a book. You need to learn by engaging the community.

Whenever you have or build top talent they always have one foot out the door. Finding a new developer takes time and requires resources from the team to vet the right candidate. Then they need to up-skill with assistance from the team. Finding and on-boarding a new developer can have a significant impact on the delivery schedule. Aligning some of the technologies/frameworks with talent in the marketplace reduces this overhead. This is probably the most frustrating driver of change. This should always be done with a cost/benefit analysis and not simply following the framework du jour.

So how do we address framework fatigue. On one hand you could choose a framework that you believe will be victorious and stick with it as long as you can. You could also manage your own framework and be responsible for everything. However I am not a betting man and don’t have the resources to keep up with every security edge-case. I would rather accept embrace change. So I externalize and break down logic as much as possible.  Additional levels of abstraction impact performance so choose wisely. Lastly spend some time between retrospectives and planning sessions to perform gap analysis between what you are doing and where the frameworks are going. If you are using Angular 1.x is the direction Angular 2 aligned with your interest? Is there another framework with an active community better suited for your needs.

Ultimately the story of living with frameworks is about living with change, choice, and constant revalidation of direction and purpose.

Running RTC, RAD, RSA, or IDA on Fedora 19.

So let me start off by saying that Fedora 19 is not a supported environment for IBM products.  That being said if you are living on the bleeding edge of technology like me you probably noticed that Rational Team Concert, Rational Application Developer, Rational Software Architect, or InfoSphere Data Architect do not work on Fedora 19 or the latest version of Ubuntu. The main reason for this is a defect in Eclipse 4.2.2 that they are built on. The resulting error looks like this:

Unhandled exception
Type=Segmentation error vmState=0x00000000
J9Generic_Signal_Number=00000004 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
Handler1=00007F88ABA54940 Handler2=00007F88AB6F4900 InaccessibleAddress=0000000000000000
RDI=0000000000000000 RSI=00007F88A4A5B5A0 RAX=00007F88A617F2C0 RBX=0000000000000000
RCX=0000003FF306DA00 RDX=0000003DE9421908 R8=0000000000000000 R9=00007F88A492DCD0
R10=00007F88AC604FA0 R11=00007F88AC6050A0 R12=0000000000000004 R13=0000000000000119
R14=00007F88ABBD0D80 R15=00007F882C5C7A2A
RIP=0000003FF306DA11 GS=0000 FS=0000 RSP=00007F88AC6053B0
EFlags=0000000000210206 CS=0033 RBP=00007F88A4A5B5A0 ERR=0000000000000004
TRAPNO=000000000000000E OLDMASK=0000000000000000 CR2=0000000000000000
xmm0 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm1 2424242424242424 (f: 606348352.000000, d: 1.385533e-134)
xmm2 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm4 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm5 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm6 00007f88ac605830 (f: 2891995136.000000, d: 6.928035e-310)
xmm7 0000000000000004 (f: 4.000000, d: 1.976263e-323)
xmm8 b1cc29b34c49b08c (f: 1279897728.000000, d: -8.161092e-69)
xmm9 116c0a02d9b0f362 (f: 3652252416.000000, d: 9.468854e-225)
xmm10 405e000000000000 (f: 0.000000, d: 1.200000e+02)
xmm11 403d5ee19101ca50 (f: 2432813568.000000, d: 2.937063e+01)
xmm12 3ee7185e8efb29c4 (f: 2398824960.000000, d: 1.101265e-05)
xmm13 3f6b2f769cf0e200 (f: 2633032192.000000, d: 3.318531e-03)
xmm14 3fe9e3779b97f4a7 (f: 2610427136.000000, d: 8.090170e-01)
xmm15 3fe41b2f769cf0e2 (f: 1989996800.000000, d: 6.283185e-01)
Module=/lib64/libsoup-2.4.so.1
Module_base_address=0000003FF3000000 Symbol=soup_session_feature_detach
Symbol_address=0000003FF306DA00
Target=2_60_20130617_152572 (Linux 3.11.2-201.fc19.x86_64)
CPU=amd64 (8 logical CPUs) (0x3d1fae000 RAM)
———– Stack Backtrace ———–
soup_session_feature_detach+0x11 (0x0000003FF306DA11 [libsoup-2.4.so.1+0x6da11])
Java_org_eclipse_swt_internal_webkit_WebKitGTK__1soup_1session_1feature_1detach+0x7f (0x00007F882C5C7AA9 [libswt-webkit-gtk-4236.so+0x5aa9])
(0x00007F88ABA6CF21 [libj9vm26.so+0x33f21])
—————————————

The defect has to do with a bug in WebKitGTK. In WebKitGTK 1.10.x a crash can occur if an attempt is made to show a browser before a size has been set. There is a fix in Eclipse 4.3 but unfortunately the IBM tooling does not build on that yet.

The first alternative to consider is to use xulrunner instead. Unfortunately only xulrunner 1.9.2 is supported for 64 bit. This is due to the fact that JavaXPCOM was removed from xulrunner 2 and later.

Unfortunately that version of xulrunner does not fully support HTML 5 nor does it reliably work under the latest Fedora or Ubuntu in my experience.  So an alternative is to upgrade the underlying Eclipse platform for those products. This is not an easy task since the products disable this capability by way of dependencies. The next best thing is to update the components of one jar.

The fix is available in the org.eclipse.swt.gtk.linux jar file in Eclipse 4.3. Here are the steps I took to resolve this issue:

  1. Install RTC, RAD, or another IBM IDE based on Eclipse 4.2
  2. Download and uncompress Eclipse 4.3 (Kepler)
  3. Open org.eclipse.swt.gtk.linux.x86_64_3.102.1.v20130827-2048.jar Eclipse 4.3 in an archive utility
  4. Open org.eclipse.swt.gtk.linux.x86_64_3.100.1.v4236b.jar from the IBM product in an archive utility.
  5. copy all of the /org/eclipse/swt/browser/WebKit*.class files from the Eclipse 4.3 swt archive to the IBM product’s archive.
  6. Start the application

Git Versus Subversion Versus Rational Team Concert – Basic Commandline Syntax Reference

It looks like Developerworks (https://www.ibm.com/developerworks) updated their theme and one of my old post is no longer usable. So I thought I would repost it in my new blog.

A while back I came across a Nick Boldt article called ” Git vs. SVN – Basic Commandline Syntax Reference.” I decided to enhance his table to include Syntax for the Rational Team Concert Command-line client.

 

Action Git Syntax Subversion Syntax Rational Team Concert Synta
Initial checkout from existing repo for a given branch git clone <url> ;cd <module>;git checkout <branch> SVN checkout <url>/<branch> lscm load -r <url>
Update locally checked out files from central repo git pull svn update lscm accept -v
List locally changes files/folders git status svn stat lscm status
 Diff locally changed file  git diff somefile.txt  svn diff somefile.txt  lscm diff file somefile.txt
 Revert locally changed file*  git checkout somefile.txt  svn revert somefile.txt  lscm undo somefile.txt
 Revert ALL local changes (except untracked files)*  git reset –hard HEAD  svn revert . -R  lscm load <workspace> -r <url>
 Add new file  git add file.txt  svn add file.txt  lscm checkin file.txt
 Add new folder recursively  git add folder  svn add folder  lscm checkin folder
 Delete file  git rm file.txt  svn rm file.txt  rm file.txtlscm checkin <parent folder>
 Delete folder  git rm -r folder (non-recursive by default; use -r to recurse)  svn rm folder (recursive by default; use -N to not recurse)  rm folderlscm checkin <parent folder>
 Commit changed file to central repo  git commit -m “<message>” file.txt; git push  svn ci -m “<message>” file.txt  lscm checkin file.txtlscm changeset comment <changeset> <message>

Legend:

  •  <URL> – Repository URL
  • <branch> – Branch or Stream or workspace
  • <module>.- the component of the repository.
  • <workspace> – Workspace is the Rational Team Concert equivalent of private Stream.
  • <changeset> – Alias or UUID of target change set.
  • <message> – Comment Text

Bringing Jazz.net into Rational Team Concert

For a while now I have been using a tool that I created called the Jazz Support Handler. I originally created it as part of a thought experiment of how to bring some of the Jazz.net experience into the Rational Team Concert client.  The tool automates searching Jazz.net and Google when an error occurs inside of eclipse. This saves me a lot of copy/pasting and additional browser windows. With the upcoming release of CLM 4.0.3 and the move to Eclipse 4.2.2, I thought I would update the dependencies and make it available to the public.

Now this tool will only catch errors passed to the ErrorSupportProvider from the Eclipse Workbench API.  it only works while the eclipse workbench is still active. Lastly this is a non-supported tool. Although I do work for IBM this is not supported by IBM nor myself. That being said I do encourage you to comment on this post with any trouble you may have.

Prerequisites

The latest bits are designed to work with the Rational Team Concert 4.0.3 Eclipse client (3.6.2 or 4.2.2 Eclipse clients). If you have RTC 4.0.3 installed in another Eclipse client than it needs to be Eclipse versions 3.6.2 or later. Eclipse also supports numerous operating systems but I have only tested it under Windows XP, Linux (Ubuntu, RHEL 6.2, and Fedora), and Apple OSX.

Installation

  1. Download the Eclipse Update Site archive from this link: JazzSupportHandlerToolUpdateSite
  2. Start your Rational Team Concert 4.0.3 Eclipse Client.
  3. Open the Install New Software Dialog (Help->Install New Software…)
  4. Select Add
  5. Archive…
  6. Point to the com.ibm.rational.jazz.support.handler.site.zip file
  7. From the Work with drop down select “All Available Sites”
  8. Select Jazz Support Handler and its child feature.
    jazz_support_handler_install
  9. Select next…
  10. Select next…
  11. Accept the license.

 Getting Started

After the install has completed there will be a new “Jazz Support Handler Category” in your Eclipse Preferences dialog.  This new preferences area is where the Jazz Support Handler can be configured and tested.

jazz_support_handler_preferencesThere are three basic settings available to the user.

  • Enabling the Jazz Support Error Handler or using the default ErrorSupportProvider.Eclipse only allows for one error handler to be enabled at a time.
  • Which data sources do you want to display? Google? Jazz.net? both? neither?
  • What do you want the tool to ignore? I am always amazed how much search engines can know about their users. I normally add filter words for my projects which prevent project specific errors from being googled.

Finally there is a test button. This is very useful for testing filters and also allows you to see the dialog without the need for an workbench error.
jazz_support_handler_example

Test Driven Development across Java, JavaScript, and Perl.

Every iteration my team explores ways of reducing time required for QA/Testing. We are a small team and do not have dedicated resources for Quality Assurance. That being said, what we can do is reduce the amount of defects found during that time. The best way to do that is to follow a Test Driven Development (TDD) methodology where you write your test from requirements before writing application code.

Now of course you cannot test everything but you can test close to one hundred percent of requirement and code coverage.  Now Test Driven Development usually is done with Unit Test but it can be done with Functional Verification Test (FVT) and Performance Verification Test (PVT) as well. However FVT and PVT require a level of requirement elaboration/documentation that is not always available.  That being said Automated Unit Test done right will reduce a large amount of possible defects.

How to perform Test Driven Development?

Now I alread described Test Driven Development (TDD) above but I thought I would go a bit deeper first before jumping into implementation across languages.  TDD is more than writing a test before code. Rather it is a repetitive iterative process of adding changes in bite sized pieces.

Test Driven Development Workflow

I recommend the following path:

  1. Start by writing a test for the smallest requirement, defect, or change request that has little to no dependencies. The smaller the change the less understanding you will need of the larger picture.
  2. Run the test you just wrote. If it succeeds then the existing code supports it and you should move on to the next smallest requirement.
  3. Write just enough code to make all of the test pass. If you have a dependency on another component that has yet to be defined or written then create an interface and mock it (I talk more about this later).
  4. Refactor your code to make it cleaner and smaller while rerunning all the test to ensure you don’t break any functionality. This also removes any dead code from retired functional requirements.

The basic test

Although different Unit Test frameworks can behave differently the “effective layout of a test case ensures all required actions are completed, improves the readability of the test case, and smooths the flow of execution. Consistent structure helps in building a self-documenting test case. A commonly applied structure for test cases has (1) setup, (2) execution, (3) validation, and (4) cleanup.

 Yes I basically stole from the Wikipedia article but I couldn’t write it any better.

Testing what doesn’t exist

I mentioned before that If you have a dependency on another component that has yet to be defined or written then you should create an interface and mock it. “Mock Objects are simulated objects that mimic the behavior of real objects in controlled ways.” It is important to note that these objects do not perform that functionality of what they are mocking. Rather they simply provide expected output for expected input.

Across three languages

In order to demonstrate how to write unit test and mock objects across three languages I decided to write the same component three times in each language.

Demonstration Projects of TDD in Java, Javascript, and Perl

The component is a simple application to read an email from a string and look up the title in an external directory (i.e. LDAP). Now the external directory doesn’t exist so I use a Mock object to test the components.

Now I am going to talk a bit about performing Unit Test, mocking, reporting coverage in Perl, JavaScript, and Java.

Unit Testing Perl

Unit testing in Perl has been around for a long time. However I often describe Perl code as being pre or post 2008. That is when Adam Kennedy gave a talk about managing a massive amount of perl  code and why testable code is important.

http://blip.tv/oslo-perl-mongers/software-testing-on-a-massive-scale-1897370

My favorite quote from the talk is “Our ability to create software is limited, primarily, by our ability to test software.” I also love the references to the city of London’s 19th century sewers. I recommend every Perl programmer watch this presentation.  You can look at any Perl code and tell if the developer used principals from this talk.

Typically I use the Test Anything Protocol for testing Perl. It has an extension for JUnit reports which allow it to fit nicely into a Rational Team Concert Development suite. I should also reference Test::Mock::Class which I use for mocking objects in Perl.

Now perl is a very dynamic language where you may only be able to identify dead code during runtime. I typically use Devel::Cover to identify dead code.

 Unit Testing in JavaScript

There are numerous testing frameworks in Javascript. Since I mostly develop using the Dojo Toolkit I use DOH.  As for Mock Objects in JavaScript there are several libraries out there. However since Javascript does not support Interfaces or Abstract Classes I have to cheat. I create stub class where each required function throws an error. For instance in one of the examples I create this interface.

/*******************************************************************************
 * Licensed Materials - Property of IBM 
 * 5748-XX8 
 * © Copyright IBM Corp.
 * 1992, 1993 All Rights Reserved US Government Users Restricted Rights - Use,
 * duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
 *******************************************************************************/
define("example/javascript/LookupInterface", [ "dojo" ], function(dojo) {
	return dojo.declare("example.javascript.LookupInterface", [], {

		/**
		 * Lookup the person's title based on their email.
		 * 
		 * @param email
		 * @return The person's title
		 * @throws AssertionError
		 *             Person could not be found.
		 */
		getTitle:function(email){
			throw new Error("Interface function getTitle(email) has not been implemented.");
		},

		// Constructor function. Called when instance of this class is created
		constructor : function() {}
	});
});

On one hand you get the benefit of an interface but the failure to implement a function is only known during runtime which is bad. As always I am open to suggestions and feedback.

Code coverage also gets a bit complicated with Javascript. I have tried extensions to Firebug and a few other tools. Unfortunately understanding Dojo separate from my code has always proved troublesome. Someone recommended JSCover but I haven’t had a chance to try it yet.

Unit Testing in Java

JUnit is possibly the most popular Unit Testing frameworks. It is also included within Eclipse by default and there are tools provided to make Test Driven Development a first class citizen. There are numerous tutorials available on the topic.

For mock objects in Java I recommend Mockito. If you are using Eclipse for Java development then I also suggest using the EclEmma. It has outstanding integration but it doesn’t report against exceptions very well.

Conclusion and follow-up topics

Test Driven Development is a very powerful tool. Looking at code writen using this method there is an obvious difference.  Code is less buggy and easier to manage. You have confidence your code is sound without having to implement the entire application. I also want to mention Jurgen Appelo excellent article on implementing a requirement across multiple iterations TDD style.

Now I purposely left some topics out for future blog post. For instance as I mentioned earlier it is possible to use Functional Verification Test (FVT), Business Verification Test (BVT). Performance Verification Test (PVT) as part of Test Driven Development. You can do more than simple Unit Test but there are more than a few gotchas.

 

Using Rational Team Concert feeds in the IBM Notes Client.

One of the great things about Rational Team Concert is it brings an awareness to a Development team. Now outside of the CLM Dashboards and the thick clients that awareness continues in the forms of email and news feeds. Now everyone knows email but when left unchecked your Inbox can become unmanageable.

My personal preference is to use the news feed capability of Rational Team Concert. Not only does it provide notifications out of my inbox but it allows me to have greater control over what I want to see. I can take any work item query and turn it into a feed.

That being said I have had trouble for some time trying to get the Feed Reader in the IBM Notes Client to read the feeds from Rational Team Concert.

Note_Feed_Authentication_Error

The problem is the default authentication mechanism in CLM is form authentication rather than basic authentication. The IBM Notes Client does not make this easy.  Here are the steps I took to enable form authentication of feeds in the IBM Notes Client.

Creating the feed in Rational Team Concert.

In Rational Team Concert you can create a feed of nearly anything. Unfortunately creating a feed from the web interface is a bit difficult but from the client is quite easy.

  1. Start by creating a Work Item query of the news you are interested.  I should mention that there are other events that can be subscribed to other than work items.
  2. Right click the query in the Team Artifacts view and select “Subscribe to Query Feed.”
    RTC_Feed_Subscribe
  3. Retrieve the URL for that feed subscription by right clicking the new feed and selecting “Copy Feed URL.”
    Retereive a URL of a query feed

Enabling Form Authentication in the IBM Notes Client

  1. First thing is to create a new account for Form Authentication In your Notes Client by selecting to
    1. File
    2. Preferences
    3. Accounts
    4. New Account
  2. Set the following information
    1. Basics
      1. Account Name: <anything>
      2. Type: HTTP/HTTPS
      3. Server: https://<Server Host Name>
    2. Log in information
      1. Select “Use direct log in or single sign on.”
      2. Enter your user name and password.
    3. Advanced Properties
      1. Authentication URL: https://<Server Host Name>/<Context Root>/service/com.ibm.team.repository.service.internal.webuiInitializer.IWebUIInitializerRestService/j_security_check
      2. Authentication Type: J2EE Form (advanced)

IBM Notes Form Authentication Settings Conclusion

Now the feed you copied from the client should work in the IBM Notes client feed reader.

Note_Feed_Authentication_Success

The beauty of this is you can now create notifications with a greater amount of flexibility. For instance you can use a single feed to notify an entire team when a new Critical defect is submitted.