TITLE OF PAPER: Agile Testing with Python Test Frameworks URL OF PRESENTATION: _URL_of_powerpoint_presentation_ PRESENTED BY: Grig Gheorghiu REPRESENTING: Avamar CONFERENCE: PyCon 2005 DATE: Wednesday, March 23, 2005 LOCATION: GWU Cafritz Conference Center, Grand Ballroom -------------------------------------------------------------------------- REAL-TIME NOTES / ANNOTATIONS OF THE PAPER: {If you've contributed, add your name, e-mail & URL at the bottom} Agile Testing with Python Test Frameworks What is agile testing Practice TDD (test-driven-development) Short feedback loop between developers and testers find bugs that matter early rather than late collaborate with programmers to improve testability shift from last line of defense to service provider focus on providing value fluid roles: do whaever job needs to be done toolsmith role: automate what you can , delivery in less than 40 hours spend time and resources only on things that provide information Involve customers early in the writing business-facing tests Deploy automated regression testing Conduct sessions of exploratory testing Python As An Agile Language Simplicity clean and simple syntax reads like pseudo-code Buitl in high level data types natrually object-oriented, encourages code reuse Communication powerful yet simple idons lets you code and oesn't get iyour way standard coding style enforced by significantwhitepspace execuatable docuemtnation FeedBack dynamci interpreted: short dev cycle, fast feedback loop interactive shell session procices instantaniuosu feeback unit test frameowkrs (unittest, doctest, py.test) enble frequent feedback courage agile testing practice: Practice TDD port of jUnit to python (un-pythonic) easy to use by people familiar with the xUnit frmeworkds test classes need to inhereit from unitest. TestCase api can get in the way Unit testing with Doctest executable documentation or literate testing no api to remember perfect way to keep documention in sync with the code can be combined with epydoc to produce "agile documentation" no provisions for test fixture management or test suite organization no api test functino /methods start with test and test claees with test save test code in files use normal assert statment setup[method| class| module] hooks provide test fixture mangement and method, pylint: pthon source code analyzier coverage accumulates vcoverage data buildbot port of MikeClark's jUnitPef adds performance testing capabilities to existing unittest tests provides 2 types of performance tests timedtest: runs an existing test case by imposing a limit to the time load test main functionality my team needs to test backup a set of files/directories/mail messages/ databases restore them verify that they are restored properly we encapsulated the functionality in a BRV class (Backup-Restore-Validate) test reulst management encapsulated in a test class logs results to stdout, logfiles, databawe http post mechanism: apache web server with firebird db backend generation of lagre data sets for load testing 100s of AD users/ exchange mailboxes (win32com, win32net) 1000s of Exchange folders/messages (CDO/MAPI/ADO STAF/STAX our goal: automated regression test on many clients/platforms from IBM: automated test distrubution, execution and reporting STAF agent runs on all the machines STAF testbed STAX service machine cordinate agents, gathers all results and logs commands are sent to STAX service host vial XML "job" files Jython code can be embedded in script elements inside XML command are executed on agents via rocess elements function definitions and calls iterate, paralleliterate, timer constructs logging and notification constructs Global Python variables vis SCRIPTFILE and SCRIPT commands Java GUI tool Used by Linux Text Project (ltp.sf.net) Nightly build completeion message sent to distrubtion lists .procmailrc file triggers Python script whichs STAX job file to the STAX service machine Script queries central log file for the new job agile testing practices involve customers in writing business-facing tests automated acceptance testing FitNesse is based on Ward Cunninghams' FIT tests as HTML tables FtNesse specific: Web server/Wiki, Test Suites, Setup/teardown pages FIT/FitNesse: test the business logic below the GUI enforce good code design practicies (MVC) create test interfaces for better code Two categoreis of test tools tools that simulate browsers (web protocol drivers) implement http request/resonse protocol some parse html into dom examples, mechanize, webunit, pbp, twill, maxQ tools that automate brosers (web browser drivesr) examples: selenium pamie, jssh, watir maxQ uses roxy to cature http requests/rpelies tool is written in java, but generates jython scripts record/layback capability captures http resonse codes and raw html can verify that a given test or URI exists in the http selenium uses real browser to play back testing scripts based on javascript: cross-platform and cross-browser tests can be written as html tables, similiar to fitnesse browser can also be driven via scripts (python ruby, perl) testrunner mode: static html, js, css pages on web site under test driven mode: standalone server (twisted-based) bypasses javascript showing an example of using selenium actualy testing a plone registration site in real-time jython number one contender in the dynamic java arena (scripting the jvm) combines agility of python with easy acccess to java libraries gvr: "testing (popular area for Jython)" java teast tools can be easily driven from within jython web app testing with htpunit from jython many test tools use jython as the test scripting language bret pettichord: "homebrew test automation" agile toolsmith: delivery something usefule in less han 40 hours huge variety of open source test tools available use high-quality scripting langnuage holistic testing unit testing with unitest doctest, py.test nightly regression testing with STAF/STAX business logic acceptance testing with PyFIT/FitNess python/agile testing blog: agiletesting.blogspot.com Q. does selenium support web apps that use frames A. it does - from what he knows there are some subtle issues with IFrames Q. how does selenium verify the test ran correctly A. it uses a DOM parser to parse the page and identify the elements to be verified if your links use id's then all is easy, otherwise you have to use xpath and the like -------------------------------------------------------------------------- REFERENCES: {as documents / sites are referenced add them below} ward cunningham's FIT: http://fit.c2.com/wiki.cgi?WelcomeVisitors selenium: http://selenium.thoughtworks.com/index.html author blog: http://agiletesting.blogspot.com/ -------------------------------------------------------------------------- QUOTES: -------------------------------------------------------------------------- CONTRIBUTORS: {add your name, e-mail address and URL below} Mike Taylor Nick Bastin (nick.bastin@gmail.com) -------------------------------------------------------------------------- E-MAIL BOUNCEBACK: {add your e-mail address separated by commas } -------------------------------------------------------------------------- NOTES ON / KEY TO THIS TEMPLATE: A headline (like a field in a database) will be CAPITALISED This differentiates from the text that follows A variable that you can change will be surrounded by _underscores_ Spaces in variables are also replaced with under_scores This allows people to select the whole variable with a simple double-click A tool-tip is lower case and surrounded by {curly brackets / parentheses} These supply helpful contextual information. -------------------------------------------------------------------------- Copyright shared between all the participants unless otherwise stated...