py.test and py lib.notes

TITLE OF PAPER: py.test and the py lib
URL OF PRESENTATION: _URL_of_powerpoint_presentation_
PRESENTED BY: Holger Krekel
REPRESENTING: ?

CONFERENCE: PyCon 2005
DATE: Wednesday, March 23, 2005
LOCATION: GWU Cafritz Conference Center, Grand Ballroom

--------------------------------------------------------------------------
REAL-TIME NOTES / ANNOTATIONS OF THE PAPER:
{If you've contributed, add your name, e-mail & URL at the bottom}

(Presenting in Keynote, so he's cool already.. :-))

Testing and Python

* automatically testing python programs is a honking great idea, let's do more ofit !
* Unfortunately, Python projects have rather disparate tests tools and "runners"
* Pythoneers deserve better than the classical unittest.py

The reason for py test is that unittest was not easy to extend and lead to him to create it

What is py.test

  * py.test is an external project idnependent tool which
    * automatically and iternatively collects tests from dirctories, files, modules and classes
      * has extensive debugging aid
      * interacts with project-specific configuration to alter the ocllection and test execution process, including adding new command line options
      * is flexible enough to allow running PyPy's tess on a different virtual terminal

py.tests features (1)

  assert with assert statement
  automatic collection of tests on all levels
  testing starts immediately
  generative tests: yielding more user-defined tests
  specify different python versions / executables
  non nterference with cmd line utilities

py.text features (2)

  debug with the print statemnt
  order of runing tests as they appear in the files
  useful trace backs, e.g. recursion detection
  Manages test state across modules, classes and methods
  it has documentation

py.tet aserting the obvious
The ide of "y.test" and the py lib in general is "no API is the best API". For etesting this e.g. means reusing the assert statement.

  def f():
     return 23

  def test_f():
    assert f() == 42

mangeing test state

setup and teardown resources at various levels

def setup_module(mod):
  mod.testfile = ...

class TestSomething:
  def setup_class(cls):
    cls.resource = ...
  def setup_method(self, method):

Generative tests
the easiets "non-api" way to extend the collection process is with generators wihcih allow to produce more tests on-the-fly

def func(arg1, arg2):
  assert arg1 == arg2 * 2

def test_more():
  for x,y in ((1,2), (2,3), (3,3)

Creative creation of tests using generators which 'yield' test functions and expected values

it's nice when it fails

py.test offers a number of helpful debugging features
  very usefule tracebacks
  isoloating print statements per test
  showing locals
  dropping int pdb on failures
  session: rerunning continuious failing test only
  --nomagic -- nocapture may be your friend :)

Configurable framework - knows about the test set size, can run just the failing tests as you edit your code until 0 tests fail.

itneractive example

he is on a live python session and running a failure demo program
  testing mode is run inprocess so it's running in the same session that py.test is running in
  showing examples of a testing config setup that allows for local changes to the test configuration
  he is showing in the output that actual test being run and also the traceback and codes on the left showing the state of the test
  he is showing an example that py.test is re-interpreting the code and results and adding to the output a note or marker on what the failure was inline so you can see it without having to enter the code editor

py.test --pdb
  when the failure happens you will enter pdb where you can debug

py.test near-future plans

py.test is to become a unified python testing tool
  refining and documenting project config
  doctests
  gui/html reporting
  distributing tests across platforms
  releasing it...

on to the py lib

py.test is part and makes use of the py lib:
  py.path: local and subversion filesystem objects
  py.execnet: ad-hoc distribution of programs
  py.magic: provides e.g. greenlets
  py.code: nicifying python introspection
  py.xml: providing simple xml/html object generation

runs on python 2.2 onwards

goals of the py.lib
  provide a high level and integrated standard set of serives and methods useful for development
  first step: project independent and flexible tesing tools
  experiment with improving python library development e.g. via explicitly exporting names
  repackageing and extending python's standard library e.g. offering a unified Path objec for both local and remote access

py lib hightligh: ad-hoc distributing programs
  py.execnet provides a smple mechanism to execute arbitrary code in remote locations
  communication between local andremote sits happen through channels

showing a live example of py.execnet

  >>> import py
  >>> py.execnet.
  * <typ>SoccketGeateway(2)
  >>> py.execnet.SshGateway('codespeack.net')
  <SshGateway 1/1 (1 active channels)>
  >>> ssh=_
  >>> ssh.remote_exec("imprt sys ; channel.send(sys.version_info)")
  <Channel id=3 open>
  >>> c=_$
  >>> c.receive()
  (2, 3, 5, 'final', 0)
  >>> ssh.remote_exec("import os; channel.send (channel.receive()+1)")
  <Channel ide=5 open>
  >>> c=_
  >>> c.send(41)
  >>> c.receive()
  42
  >>> c
  <Channel ide=5 closed>
  >>> c.receive()
  Traceback happend because the channel is closed
  started showing other process's that are also running and showed how to connect to another computer to show that it is distributed

  >>> ssh.remote_exec("""
    .... while 1:
    ....    fn = channel.receive()
    ....    channel.send(open(fn).read())
    .... """)

    shows a huge listing of filenames that were on the remote system

future (and thanks for) fish

finalize design of py.test and include doctests
refine consistency and actually release the damn ting
expose remote Path-Over-Ssh objects
try improve windows-interactions

py lib info

the py lib is driven by Holger Krekel and Armin Rig

Q. is the py.test all python
A. it is complete python but there is a directory of c extensions that could be compiled if you have access to a compiler

Q. do you use py.test for acceptance testing or do you limit yourself to unit tests
A. to be honest he can't follow the whole seperation of unittests from acceptance testing - the example he showing is an example of testing the documentation that uses a generator that generates the rest test function to see if the docs are generated properly

--------------------------------------------------------------------------
REFERENCES: {as documents / sites are referenced add them below}
py.test at <http://codespeak.net/py/current/doc/test.html>
py lib at <http://codespeak.net/py
py-dev@codespeak.net

--------------------------------------------------------------------------
QUOTES:

--------------------------------------------------------------------------
CONTRIBUTORS: {add your name, e-mail address and URL below}
Nick Bastin (nick.bastin@gmail.com) <www.nickbastin.com>
Mike Taylor (he wrote most of it)

--------------------------------------------------------------------------
E-MAIL BOUNCEBACK: {add your e-mail address separated by commas }

--------------------------------------------------------------------------
NOTES ON / KEY TO THIS TEMPLATE:
A headline (like a field in a database) will be CAPITALISED
    This differentiates from the text that follows
A variable that you can change will be surrounded by _underscores_
    Spaces in variables are also replaced with under_scores
    This allows people to select the whole variable with a simple double-click
A tool-tip is lower case and surrounded by {curly brackets / parentheses}
    These supply helpful contextual information.

--------------------------------------------------------------------------
Copyright shared between all the participants unless otherwise stated...