TITLE OF PAPER:         ATOP - Simple but Powerful Persistence using Python and BSDDB

URL OF PRESENTATION:    _URL_of_powerpoint_presentation_
PRESENTED BY:           Glyph Lefkowitz <glyph@divmod.com>
REPRESENTING:           Divmod

CONFERENCE:             PyCON 2004
DATE:                   20040324
LOCATION:               Amphitheater

--------------------------------------------------------------------------
REAL-TIME NOTES / ANNOTATIONS OF THE PAPER:
{If you've contributed, add your name, e-mail & URL at the bottom}

An object database built on top of Berkley DB.

Its suppose to be small and simple to install.  It is designed to be an object repository for a Personal Information Management system.

ATOP - Atomic Transactional Object Persistence System

Atomic
    "All or nothing"
    Allows you to do half of an operation and not have it count.

Transactional
    Allows you to back out half of an operation.
    "Safe" i.e. if your code breaks, your database doesn't.

Object
    An object database (not a relational database).
    Designed for flexibility/feature growth
    Simple representations of relationships between objects.
    Aggregations (containers).

Persistence
    Objects stay around, don't disappear when your program crashes.
    The objects are just there, always around, always in storage (i.e. not actively stored on disk or retrieved from there).

Motivation
    Flexibility (lots of easy upgrades)

Things they've tried
    Metakit
        Very fast to a point
        Bugs in the binding
        Coredumping
    Custom Object-relational database (TwistedWorld)
        It did "everything"
        Too complicated; lots of work
        Too slow: 1 transaction in 3 seconds
    Simple Filesystem Storage
        Worked great to a point (just like Metakit)
        Too slow again
        Not enough granularity

Didn't try
    ZODB
        too much magic
        ZODB 2 vs 3
        lots of code dependency
    PySQLite
        great for traditional, conservative RDBMS work
        nondeterministic performance for other use
    Cog
        not geared toward large databases
    xsdb
        not ready at the time

Solution: ATOP
    Keep it simple but get all the features needed
    Use performant lower level (BSDDB)
    Use pickle, no custom data packing

ATOP fundamental concept
    Store
        the database
        holds items and files (external data)
    Item
        A thing in the database/store
        Objects must subclass Item, or use Item directly
        Items have numeric IDs -- no semantic URL structures
    Pool
        Large, indexed collection of items
        Something like a "table"
        Items live in the store, not in a pool
        Pools contain items
        Pools are items that have indexes
        Pools can be queried

        pool = store.getItemByID(7)
        for item in pool.queryIndex( 'name', startKey='Bob'):
            print item.name

Tabular Data Browser
    Allows for a tabular data browser (web app)
    Getting this right and fast was the entire goal
    This is difficult using an RDBMS due to differing query requirements, indexes, etc.

    Performance ~= constant time for queries (or no worse than usual py experience)

BSDDB
    Integrated with Python
    Available on Windows, no install
    Reliable, performant, well-supported
    But extremely difficult to use correctly
        Failure modes are dangerous
        Can corrupt if
            opened incorrectly
            manipulated when not recovered
            one process crashes and others don't stop immediately
            fail to close the database cleanly
            fail to open with recovery when necessary
    ATOP takes care of all of this for you

Questions
Q: Did you try filesystem storage on top of ReiserFS?
A: Yes, tried it, syscalls get in the way. Also, Win32 is a target platform, among others.
Q: Is there caching?
A: Yes, but doesn't handle concurrent access yet, working on it. Cross-object references take advantage of it.

--------------------------------------------------------------------------
REFERENCES: {as documents / sites are referenced add them below}

--------------------------------------------------------------------------
QUOTES:

atop:        http://www.divmod.org/Home/Projects/Atop/

--------------------------------------------------------------------------
CONTRIBUTORS: {add your name, e-mail address and URL below}
Bob Kuehne, rpk@blue-newt.com, http://www.blue-newt.com
Abhay Saxena, ark3@email.com

--------------------------------------------------------------------------
E-MAIL BOUNCEBACK: {add your e-mail address separated by commas }



--------------------------------------------------------------------------
NOTES ON / KEY TO THIS TEMPLATE:
A headline (like a field in a database) will be CAPITALISED
    This differentiates from the text that follows
A variable that you can change will be surrounded by _underscores_
    Spaces in variables are also replaced with under_scores
    This allows people to select the whole variable with a simple double-click
A tool-tip is lower case and surrounded by {curly brackets / parentheses}
    These supply helpful contextual information.

--------------------------------------------------------------------------
Copyright shared between all the participants unless otherwise stated...