Profiling and Visualizing Python.notes

Thursday, March 24, 2005

TITLE OF PAPER: Python Profiling and Visualization
URL OF PRESENTATION: _URL_of_powerpoint_presentation_
PRESENTED BY: Richard Saunders
REPRESENTING: Rincon Research Corporation

CONFERENCE: PyCon 2005
DATE: Thursday, March 24, 2005
LOCATION: GWU Cafritz Conference Center, Grand Ballroom

--------------------------------------------------------------------------
REAL-TIME NOTES / ANNOTATIONS OF THE PAPER:
{If you've contributed, add your name, e-mail & URL at the bottom}

Background
    rincon does DSP, ffts, fliters, demods, etc.
    See "An empirical comparison of C, C++, Perl, Python, Rexx, and Tcl", IEEE
        Computer
    There are profiling hooks built into Python; you can arrange for them to
        call any function you like. There's a 'profile' module.

tried
 - profile
 - hotspot

both deterministic

Approach
 - Needs steady state debuggable applications

 - Site tunable: want to toggle profile while running: dynamic prof
 - minimal intrusion: cheap for prod code

2-prong approach:
    Wrote something like "top" for Python
        This uses a custom instruction available only on x86 that is a cycle
            counter to timestamp the function calls.
        Top runs on a separate CPU (so it doens't throw things off) and
            communicates with the Python interpreter via shared mem.
    Existing Visualization tools

Python top
    Found they should optimise writestr
    Reports time in function vs. time in called functions
    Uses multi-CPU environments to offload the profiling computations from
        target program


9:
ULMA: a monitoring framework
    hooks desecribe user code perf, but not pyvm perf
    
Hooks into PyDict_Type dictionary, replacing tartget routine pointer with
        monitor path code pointer
    No overhead for non-monitored methods
    VM code not touched

 Unicon representation: hexigon file represents growth of the stack
 
 Nova representation: Spin clocks represent time spent in task

    Color represents the type of data being manipulated, ex: list, dict, ...
    Length of clock hand indicates size of object at construction.

    
Example of using Algae and other tools to optimise a little program
    Found that Tkinter was making a lot of dictionaries

(all slides are in references, in the pdf)



q&a:
 - memory prof? yes, alamo can do it
 - web prof? uh, yes. it's an app, right? of course you can profile it.
 - i/o prof? not explicitly, but should be obvious, for example, doing lots of 'write' blocking
 - top portable? yep.


--------------------------------------------------------------------------
REFERENCES: {as documents / sites are referenced add them below}
"An empirical comparison of C, C++, Perl.. ": http://page.mi.fu-berlin.de/~prechelt/Biblio/jccpprtTR.pdf
talk slides:  http://www.python.org/pycon/2005/papers/49/PyCon2005ProfilingandVisualizing.pdf
http://unicon.sourceforge.net/

--------------------------------------------------------------------------
QUOTES:



--------------------------------------------------------------------------
CONTRIBUTORS: {add your name, e-mail address and URL below}
Nick Bastin <nick.bastin@gmail.com> <http://www.nickbastin.com>
Bob Kuehne <rpk@blue-newt.com> <http://www.blue-newt.com>
Andy Wright <waw@well.com>
Erik Rose <corp@grinchcentral.com>

--------------------------------------------------------------------------
E-MAIL BOUNCEBACK: {add your e-mail address separated by commas }



--------------------------------------------------------------------------
NOTES ON / KEY TO THIS TEMPLATE:
A headline (like a field in a database) will be CAPITALISED
    This differentiates from the text that follows
A variable that you can change will be surrounded by _underscores_
    Spaces in variables are also replaced with under_scores
    This allows people to select the whole variable with a simple double-click
A tool-tip is lower case and surrounded by {curly brackets / parentheses}
    These supply helpful contextual information.

--------------------------------------------------------------------------
Copyright shared between all the participants unless otherwise stated...