Agile Testing with Python Test Frameworks.notes

Wednesday, March 23, 2005

TITLE OF PAPER: Agile Testing with Python Test Frameworks
URL OF PRESENTATION: _URL_of_powerpoint_presentation_
PRESENTED BY: Grig Gheorghiu

DATE: Wednesday, March 23, 2005
LOCATION: GWU Cafritz Conference Center, Grand Ballroom

{If you've contributed, add your name, e-mail & URL at the bottom}

Agile Testing with Python Test Frameworks

What is agile testing

    Practice TDD (test-driven-development)
    Short feedback loop between developers and testers
      find bugs that matter early rather than late
      collaborate with programmers to improve testability
      shift from last line of defense to service provider
    focus on providing value
      fluid roles: do whaever job needs to be done
      toolsmith role: automate what you can , delivery in less than 40 hours
      spend time and resources only on things that provide information
    Involve customers early in the writing business-facing tests
    Deploy automated regression testing
    Conduct sessions of exploratory testing

Python As An Agile Language
  clean and simple syntax reads like pseudo-code
  Buitl in high level data types
  natrually object-oriented, encourages code reuse
  powerful yet simple idons
  lets you code and oesn't get iyour way
  standard coding style enforced by significantwhitepspace
  execuatable docuemtnation
  dynamci interpreted: short dev cycle, fast feedback loop
  interactive shell session procices instantaniuosu feeback
  unit test frameowkrs (unittest, doctest, py.test) enble frequent feedback
agile testing practice: Practice TDD
port of jUnit to python (un-pythonic)
easy to use by people familiar with the xUnit frmeworkds
test classes need to inhereit from unitest. TestCase
api can get in the way

Unit testing with Doctest
executable documentation or literate testing
 no api to remember
 perfect way to keep documention in sync with the code
 can be combined with epydoc to produce "agile documentation"
 no provisions for test fixture management or test suite organization
no api
  test functino /methods start with test and test claees with test
  save test code in files
  use normal assert statment
setup[method| class| module] hooks provide test fixture mangement and method,

pylint: pthon source code analyzier
  accumulates vcoverage data


port of MikeClark's jUnitPef
adds performance testing capabilities to existing unittest tests
provides 2 types of performance tests
  timedtest: runs an existing test case by imposing a limit to the time
  load test

main functionality my team needs to test
  backup a set of files/directories/mail messages/ databases
  restore them
  verify that they are restored properly
  we encapsulated the functionality in a BRV class (Backup-Restore-Validate)

test reulst management encapsulated in a test class
  logs results to stdout, logfiles, databawe
  http post mechanism: apache web server with firebird db backend
generation of lagre data sets for load testing
  100s of AD users/ exchange mailboxes (win32com, win32net)
  1000s of Exchange folders/messages (CDO/MAPI/ADO

our goal: automated regression test on many clients/platforms
from IBM: automated test distrubution, execution and reporting
  STAF agent runs on all the machines STAF testbed
  STAX service machine cordinate agents, gathers all results and logs
  commands are sent to STAX service host vial XML "job" files
  Jython code can be embedded in script elements inside XML
  command are executed on agents via rocess elements
  function definitions and calls
  iterate, paralleliterate, timer constructs
  logging and notification constructs
Global Python variables vis SCRIPTFILE and SCRIPT commands
Java GUI tool
Used by Linux Text Project (

Nightly build completeion message sent to distrubtion lists
.procmailrc file triggers Python script whichs STAX job file to the STAX service machine
Script queries central log file for the new job

agile testing practices
  involve customers in writing business-facing tests
  automated acceptance testing
FitNesse is based on Ward Cunninghams' FIT
  tests as HTML tables
  FtNesse specific: Web server/Wiki, Test Suites, Setup/teardown pages
  FIT/FitNesse: test the business logic below the GUI
    enforce good code design practicies (MVC)
    create test interfaces for better code

Two categoreis of test tools
tools that simulate browsers (web protocol drivers)
  implement http request/resonse protocol
  some parse html into dom
  examples, mechanize, webunit, pbp, twill, maxQ
  tools that automate brosers (web browser drivesr)
  examples: selenium pamie, jssh, watir

maxQ uses roxy to cature http requests/rpelies
tool is written in java, but generates jython scripts
record/layback capability
captures http resonse codes and raw html
can verify that a given test or URI exists in the http

  uses real browser to play back testing scripts
  based on javascript: cross-platform and cross-browser
  tests can be written as html tables, similiar to fitnesse
  browser can also be driven via scripts (python ruby, perl)
  testrunner mode: static html, js, css pages on web site under test
  driven mode: standalone server (twisted-based)
    bypasses javascript

 showing an example of using selenium actualy testing a plone registration site in real-time

number one contender in the dynamic java arena (scripting the jvm)
combines agility of python with easy acccess to java libraries
gvr: "testing (popular area for Jython)"
java teast tools can be easily driven from within jython
  web app testing with htpunit from jython
many test tools use jython as the test scripting language

bret pettichord: "homebrew test automation"
agile toolsmith: delivery something usefule in less han 40 hours
huge variety of open source test tools available
use high-quality scripting langnuage

holistic testing
  unit testing with unitest doctest, py.test
  nightly regression testing with STAF/STAX
  business logic acceptance testing with PyFIT/FitNess
python/agile testing blog:

Q. does selenium support web apps that use frames
A. it does - from what he knows there are some subtle issues with IFrames

Q. how does selenium verify the test ran correctly
A. it uses a DOM parser to parse the page and identify the elements to be verified
   if your links use id's then all is easy, otherwise you have to use xpath and the like

REFERENCES: {as documents / sites are referenced add them below}

ward cunningham's FIT:
author blog:       


CONTRIBUTORS: {add your name, e-mail address and URL below}
Mike Taylor
Nick Bastin ( <>

E-MAIL BOUNCEBACK: {add your e-mail address separated by commas }

A headline (like a field in a database) will be CAPITALISED
    This differentiates from the text that follows
A variable that you can change will be surrounded by _underscores_
    Spaces in variables are also replaced with under_scores
    This allows people to select the whole variable with a simple double-click
A tool-tip is lower case and surrounded by {curly brackets / parentheses}
    These supply helpful contextual information.

Copyright shared between all the participants unless otherwise stated...