Testing Command-line Applications with Aruba

Cucumber is often used to test web applications. Many developers hook it into their Rails projects to integration test site features. Wouldn’t it be great if there were a way to test command-line applications in a similar fashion? You can with Aruba.

Aruba

Aruba is a Cucumber extension for testing command-line applications written in any language. Passing arguments, interacting with the file system, capturing exit codes, and mimicking interactive usage are all features provided out of the box. Below is a basic test for the mv command that passes:

Scenario: Backing up test.conf
  When I run `mv test.conf test.conf.bak`
  Then the output should contain:
  """
  mv: rename test.conf to test.conf.bak: No such file or directory
  """

Now let’s showoff a few of Aruba’s built-in steps to prevent the command from failing:

Scenario: Backing up test.conf
  Given an empty file named "test.conf"
  When I run `mv test.conf test.conf.bak`
  Then the exit status should be 0
  And the following files should exist:
    | test.conf.bak |
  And the following files should not exist:
    | test.conf     |

The first step creates an empty file and executes mv inside of Aruba’s sandbox directory. After the mv command is executed, its exit status is compared to 0 and the existence of test.conf.bak (and non-existence of test.conf) is confirmed.

It’s also worth noting that after each scenario Aruba clears out its sandbox — a temporary directory that becomes the current working directory for your command-line tool — unless you explicitly tag the scenario with @no-clobber. This tag preserves the previous scenario’s final state. Tying this back to the example above, the next scenario would begin with only test.conf.bak in the sandbox. Additional Aruba-specific tags can be found in the README.

Extending the Aruba API

As a command-line application evolves, other conditions not available in Aruba’s built-in API will require testing. For example, say you need to assert a file’s user and group attributes. Because Aruba’s API was built using Ruby modules, it can be reopened inside of Cucumber’s env.rb:

module Aruba
  module Api
    def check_file_owner_and_group(paths_and_users_and_groups)
      prep_for_fs_check do # Lower-level function provided by Aruba
        paths_and_users_and_groups.each do |path, user, group|
          stat = File.stat(path)

          Etc.getpwuid(stat.uid).name.should == user
          Etc.getgrgid(stat.gid).name.should == group
        end
      end
    end
  end
end

Then create a matcher:

Then /^the following files should have username "([^"]*)" and group "([^"]*)":$/ do |user, group, files|
  check_file_owner_and_group(files.raw.map { |file_row| (file_row << user) << group })
end

Now that step can be included to test the user and group attributes of files:

Scenario: Backing up test.conf
  Given an empty file named "test.conf"
  When I run `mv test.conf test.conf.bak`
  And the exit status should be 0
  And the following files should exist:
    | test.conf.bak |
  And the following files should not exist:
    | test.conf     |
  And the following files should have username "hector" and group "staff":
    | test.conf.bak |

Conclusion

Using a behavior-driven development approach for building command-line applications with Cucumber and Aruba was a pleasure. Aruba’s API covers a decent amount of ground and was easily expandable. The source code was straightforward and after skimming its internals, I was able to expand the API to meet my needs. Hopefully reading this will help you do the same.

My name is Hector Castro and I live in Philadelphia, PA. You can see some of my public projects on GitHub, and if you're old-fashioned here is my resume.

You can get in contact with me via Twitter or E-mail.