BlackhawkDocs

From Eigenpedia

Jump to: navigation, search

This is the top-level wiki page for the Blackhawk test-control harness project (aka TCH, aka coconut, and under incubation as part of The Eigenbase Project).

Contents

[edit]

What is it?

Blackhawk is a pluggable test harness that allows for dynamic composition of test suites based on test meta-data, parameterization of test cases, and multi-process management. It supports junit, a native test framework similar to junit called "javatest," and has support for http protocol testing. The pluggable interface allows for other test harnesses to be plugged in, such as junit and pyunit.

[edit]

Current Status

This project is currently in incubation. The source code has been transferred from Apache's Beehive project into Eigenbase. Short term goals are basically to clean up the build/test environment so that development can occur, and inventory features that should be added and subtracted or cleaned up. Usability is a high priority area, especially in simplifying the syntax and usage of the framework.

[edit]

Goal

The goal of Blackhawk is to make JUnit (and other frameworks) be able to go beyond simple unit testing. Unit testing often consists of simple method calls to the software under test. What Junit does not address are things such as:

  • parameterization of tests in order to cover a test matrix
  • dynamic suite composition
  • test meta-data, i.e. attributes, so that suites can be composed of tests with a given set of attributes
  • specification of dependencies (i.e. this test must be run before that test)
  • test isolation (if my test case crashes, i don't want my test harness to crash with it)
  • composite test scenarios (e.g. a test is defined as these three tests run in parallel; result of the composite test is a logical AND of the results of the three tests)
  • multi-process test management (e.g. i want to bring up three servers, make A send messages to B with C as a proxy, I want to shut down C in the middle of sending, and check if A handles the broken connection properly. This scenario, I want to implement inside my test method)


When Blackhawk is released, it should be able to take existing junit tests and run them inside the harness. And allow developers to utilize any of the above mentioned features mentioned above. But for all Junit tests that do not require the above, that they be able to run just as before.

[edit]

Get the Source Code

If you just want to look through the codebase, you can use p4web


Otherwise, to get the latest code from the Eigenbase Perforce server:

  • P4PORT=perforce.eigenbase.org:1666
  • P4USER=your-sourceforge-user-name
  • P4CLIENT=your-user-name.your-hostname

In order to make changes, send your username to the Blackhawk project admin and request write access. For read-only clients, use the predefined username "guest".

Once you've got Perforce set up, run p4 client. Map your view: 

 //open/blackhawk/... //YOURCLIENT/blackhawk/...

Finally, run p4 sync to get all source files.

[edit]

HOWTO

[edit]

Use Metadata To Control Whether a Test Runs On a Particular Operating System

Run on Windows or Linux only: 

      <test name="t1">
        <test-metadata>
          <os>win lin</os>
        </test-metadata>
        <javatest testclass="org.eigenbase.blackhawk.internal.test.EmptySuccessTest"/>
      </test>

Currently supported values:

  • win: Windows
  • lin: Linux
  • sol: Solaris
[edit]

Pass parameters from test.xml to an ant task

  • When calling an ant task from the test.xml something like: 
<project name="bh" default="target1">
  <target name="target1"/>
    <test-suite name="suite1">
      <test-parameter name="myVar" value="100" />
      <test name="test1">
        <ant-task>
          <ant antfile="build.xml" target="my-ant-target" />
        </ant-task>
      </test>
    </test-suite>
  </target>
</project>
  • ...values defined as test-parameters ("myVar" in the example) can't be accessed from from the ant task.
  • Instead, declare them as a property, reference the property to set the test-parameter, and the property will also be visible in the ant task. 
<project name="bh" default="target1">
  <property name="myVar" value="100"/>
  <target name="target1"/>
    <test-suite name="suite1">
      <test-parameter name="myVar" value="%{myVar}" />
      <test name="test1">
        <ant-task>
          <ant antfile="build.xml" target="my-ant-target" />
        </ant-task>
      </test>
    </test-suite>
  </target>
</project>
[edit]

TODO

  • tag all source code with apache copyright and license notices
  • repackage all source as as org.eigenbase...
  • remove "infra" directory
  • simplify the build to remove all "beehive" artifacts
  • top level targets should be:
    • build (main compile of code)
    • test (run all tests)
    • dist (create distribution - unzipped)
    • package (zip distribution)
    • deploy (eventually if other eigenbase projects use blackhawk, deploy into the common third party area)
[edit]

Random Stuff

  • TchOklahoma: discussions on how to come up with a framework that can be loved by both developers and QA engineers
  • TchResearch: some related projects
  • bh-feature-ideas
  • bh-datamodel-improvement
  • bh-usecases

Retrieved from "http://pub.eigenbase.org/wiki/BlackhawkDocs"