Switching to Linux Mint

This is a description of my migration from Ubuntu to Linux Mint 14.  As this was my work laptop, there are some additional requirements that need to be met by the system.  In particular, I review:

  • Configurability and ease of use
  • Hardware support for a ThinkPad T430s
  • Printer and display configuration
  • Full disk encryption
  • Running Windows in VirtualBox

Continue reading

Posted in Linux, Linux Mint, VirtualBox | Tagged , , , , , , , , | 1 Comment

No-nonsense ASCII chart

There are a lot of ASCII charts floating around the Internet.  However all of them seem to have a few fundamental problems:  They include the whole ASCII set in the chart and cram too much information of each char, making the chart illegible.  What I want is a clean, readable chart that I can glance at from a distance.

Therefore I made my own, the No-nonsense ASCII chart.  It features:

  • the hex code and character, easily legible from two meters away
  • the decimal code, since this is occasionally needed
  • only the control characters I have ever heard about  (NUL excluded – everyone knows that anyway)
  • numbers and letters as a range, not individually printed out
  • no nonsense

The chart is available in PDF and OpenDocument formats sized for A4 and Letter.  It’s licensed as CC-BY.  Customize as you will.

I recommend highlighting the most often used characters with a yellow highlighting pen (for web developers %, &, ? and =).

ASCII chart

Posted in Coding | Tagged | Leave a comment

Android Torture

Okay, this is my first rant post.  It’s 5:30 in the morning and I’m writing a blog post thanks to my new-and-shiny Samsung Galaxy S3.

I was recently force-upgraded from my Nokia E7 because its USB-adapter broke off – again.  Long story short, I’m now adapting to life as a new Android user.

One of the most frustrating things immediately visible is how every application on the device believes it’s the center of the universe craving for attention.  Even though I haven’t installed many apps, my notification board is often crammed with stuff.  Right out of the box the phone had started beeping to the beat of incoming email, and it took me quite a while to figure out how to turn it off.

But tonight has been a nightmare.

At around 4 AM the phone bleeped.  I probably hardly stirred.  Then it bleeped again.  And again.  And again.  Slowly but surely it was wrenching me from sleep.

Continue reading

Posted in Android | Tagged , , | Leave a comment

Documenting Cucumber steps

One of the great things in Cucumber testing is that as you go along, you start to accumulate ready step definitions, which makes writing new tests even faster.  Several times I’ve actually been surprised that a definition for a step I wrote already existed and worked as I expected it to.

However, I noticed that developers in our project were also writing duplicate step definitions, because they didn’t know the definition already existed with a different name.  We defined some scenario writing guidelines and nomenclature to be used, which helped somewhat.  Still, in the mass of test code it was often hard to find whether the step you needed already existed.

Things like Wally and Relish are great for converting the feature files into pretty documentation, but I couldn’t find anything similar for the step definitions themselves.  So I wrote my own:  cuke-steps

The script parses the step definition files and generates a single page document containing all the step definitions it finds, in a pretty-printed fashion.  By clicking on any definition you can see the file and row number where the definition is and the code for the step.

Originally I wrote the script to create Confluence wiki markup.  We have a Jenkins job run every night that generates the documentation and pushes it into our wiki using the Confluence Publisher Plugin.  I wrote a simple HTML outputter when publishing the script.  You can check out a sample output page generated from Alister Scott’s wmf-custom-page-object.

There’s still some work to be done, like prettifying the HTML output, creating a gem out of it and making it link directly to a source repo.  Any updates are appreciated.

Posted in Cucumber, Testing | Tagged , , | Leave a comment

Dump World on failure

One of the most important aspects of test reports is that when something fails, you can easily debug what went wrong.  Splitting tests into logical, small tests that exercise a single functionality in the system goes a long way toward this goal.  Inevitably, you still occasionally get a weird failure where you have no idea what’s going wrong.

In such circumstances it’s vital to have sufficient information included in the test report to understand what happened — otherwise you’re stuck trying to do an autopsy without a corpse.  When running Cucumber tests, we have found that a dump of the World is often extremely useful information.

The following piece of code does just that.  Whenever a Cucumber scenario fails, it dumps the contents of the World to the console:

After do |scenario|
  if scenario.failed?
    Kernel.puts "Scenario failed, world contents:"
    vars = self.instance_variables.select { |v| !v.to_s.include?("__") }
    vars.sort.each do |v|
      value = instance_variable_get(v)
      Kernel.puts "    #{v}=#{value}"
    end
  end
end

This iterates through the variables and writes them out one by one.  The select statement on line 4 filters out some Cucumber-specific variables stored in the World.

You can easily customize this to output elsewhere (we’ve found Kernel.puts to be the most reliable way to get the info out) or how to output certain data types.  For example, we use the rest_client gem extensively, so we’ve customized the hook to detect the response object and to output value.code, value.headers and value.body in that case.

Posted in Coding, Cucumber, Testing | Tagged , , , | 1 Comment

API testing with Cucumber

In our project we have a REST API using JSON that is used both internally by our mobile clients and offered externally to third parties.  Since we were consolidating all functional testing to Cucumber, it was natural to experiment what would be the best way to test API’s in Cucumber.

The common first instinct is to write out the requests, payload and response expectations directly in the feature files.  After all, a major point of Cucumber and ATDD is specification by example.  The problem is that this results in very verbose descriptions that are hard to follow.

Consider the following example given in the json_spec gem:

Feature: User API
  Scenario: Index action includes full user JSON
    Given the following user exists:
      | id | first_name | last_name |
      | 1  | Steve      | Richert   |
    And I visit "/users/1.json"
    And I keep the JSON response as "USER_1"
    When I visit "/users.json"
    Then the JSON response should be:
      """
      [
        %{USER_1}
      ]
      """

Consider that for a moment.  Do you understand what’s happening in this use case?  Next imagine giving this over to your business owner for a feature review.

One of the goals of Cucumber and specification by example is to provide a common language between developers and business. The above example contains so much detail that the intent of the use case is lost.  The code reminds me a lot of an example that Aslak Hellesøy gave of using the web_steps.rb “training wheels” originally included in the Cucumber-Rails gem:

Scenario: Successful login
  Given a user "Aslak" with password "xyz"
  And I am on the login page
  And I fill in "User name" with "Aslak"
  And I fill in "Password" with "xyz"
  When I press "Log in"
  Then I should see "Welcome, Aslak"

This kind of detail should be abstracted out of the scenarios into higher-level steps and by use of page objects.  Such refactoring makes the tests read better and much less brittle.

So if you don’t write explicit operations, what do you write?  Consider what a business owner would write as individual requirements.  Describe the intent of the API, not the explicit details.

I came up with a quite nice format in which to write the API tests.  As an example of registration:

Scenario:  Successful registration
   When I perform registration with the required parameters
   Then the request should be successful
    And I should receive a valid access token

This describes the essential functionality of the API, without any of the gory details.  The corresponding step definitions could be something like:

When /^I perform registration with the required parameters$/ do
  # Populate request information
  @request_method = "POST"
  @request_url = "/rest/register"
  @request_body = {email: generateEmail, password: "pass123",
      terms_accepted: true}
  @successful_response_code = 201
  @error_response_code_email_reserved = 409
end

Then /^the request should be successful$/ do
  # Perform the request and assert the response code
  @response = RestApi.call(@request_method, @request_url,
      body: @request_body, expect: @successful_response_code)
end

Then /^I should receive a valid access token$/ do
  # ... test that the token in @response is valid
end

The interesting part is that the actions (“When”) store fields to the World, and only the post-condition “Then the request should be successful” performs the request.  This allows adding more directives that modify the request, resulting in very natural language:

Scenario:  Failed registration; email address in use
   When I perform registration with the required parameters
    But the email address is already reserved
   Then the request should fail due to email reserved

Here the step definition for “But the email address is already reserved” would either register the email address in @request_body.email or set the email address to something that is known to be reserved.  The corresponding “should fail” step expects the response code in @error_response_code_email_reserved.

This approach also abstracts the setting up the request parameters.  When we needed to make a change to the mandatory registration parameters, there was only one method that needed to be changed.

This structure has been a great success in our project.  Very often I’ve noticed that after having the feature file defined, it takes only 5-10 minutes to implement the steps.  There’s a lot of step reuse, and often you only need to write the request setup step, and possibly some custom post-condition steps.

One caveat of this approach is that the exact methods, URLs and parameter names are not presented in the feature file, and thus it does not function as documentation on its own.  We have a separate API document, and we’ve been considering moving the corresponding parts to the free-text description area in the feature files.  This would not be executable specification, but I think it’s an acceptable compromise.  I feel it more important to keep the scenarios succinct and readable, and thus more maintainable.

Posted in Coding, Cucumber | Tagged , , | 3 Comments