The State of Python.notes

Thursday, March 24, 2005

TITLE OF PAPER: The State of Python
URL OF PRESENTATION: _URL_of_powerpoint_presentation_
PRESENTED BY: Guido van Rossum
REPRESENTING: Elemental Security

CONFERENCE: PyCon 2005
DATE: March 24, 2005
LOCATION: Marvin Auditorium
--------------------------------------------------------------------------

REAL-TIME NOTES / ANNOTATIONS OF THE PAPER:
{If you've contributed, add your name, e-mail & URL at the bottom}

"Keynotes suck because I don't get to talk about the details of language design"

Brief adve
rtisement for Elemental Security
    public launch April 5th
    We will hire you please

Why My Keynotes Suck
    I just have this feeling that something is missing (blank slide)

What
's with the beard?
    Beta 'crunch' beard
    At Elemental I'm now called "professor"

Decorators Redux
    5 variations splicing [decorator, ...] into a def

    So what happened? How did @deco win?
        It's the one that everybody dislikes equally :-)
        It's unambiguous and uses prefix syntax
        Java 1.5 has similar syntax
    Why is it so controversial?
        If you can handle __init__ why is @deco so bad?
     
        Get over the Perl similarity already!

    That
's just the way it's going to be in 2.4; deal with it.

Why no class decorators?
    Superfluous given metaclasses
    The need to be at near the top is easily met witout adding new syntax.
    May be revisited if someone cares enough
        Needs motivating use cases / examples

Python's Growing Popularity
    14% (from 8%) -InfoWorld Survey

        "
But the big winner this time around is the object-oriented scripting language Python, which saw a 6 percent gain in popularity, almost doubling last year's results." --www.infoworld.com/article/04/09/24/39FErrdev_1.html
    Burton
Group report on "P-languages" (Perl, Python, PHP)
        "Both Perl and Python have been used in application development and for limited integration, but it appears that Python is a better fit for this domain for several reasons. Python is easy to learn, fully functional, syntactically clean, and very modular." --report available for $$$ via www.burtongroup.com


Jolt Productivity Award

Python.org Growth
    
Feb 2004 python.org graffic:
        793K visits from 421K sites (1.0TB)

    Feb 2005 python.org traffic:
        1023K visits from 473K sites (1.3 TB)
    Growth in one year:
        visits +29% originating sites +12%, data +30%
        and Feb 2004 had a leap day ;-)
    There is room for growth, but there is growth. Next year should be much
        more because the new site will be out.

Python Security Response Team

    Python's first official vulnerability Alter
        Accidental back door in SimpleXMLRPCServer
        Fixed in Python 2.3.5, 2.4.1
    Released the need for a standard way to handle vulnerabilities
    Formed Python Security Response Team
    security@python.org

    So far, received only a small amount of spam ;-)
    Burton Group: P
ython has fewer vulnerabilities than Perl or PHP.

Release Talk
    It's official: no new features in double-dot releases
    2.4.1: rc2 just out; final right after PyCon
        (sez Anthony Baxter)

    
2.5: next year?
    3.0 (a.k.a. "Python 3000"): Who knows? This wil
l be the first release to
        majorly break compatibility: perhaps by adding keywords, changing built-
        ins, etc.
    Reminder: slow growth policy



Python 3000
    To be clear: this is the same as Python 3.0
    Some progress: collecting ideas in PEP 3000
    Py3k is not going to be "second system syndrome done right" (Larry Wall)
    In Particular, not rewriting CPython from scratch
    More likely, various Py3k features will show up in Python 2.5, 2.6, ...; some
        
directly, some with a __future__ import
    Python 3.0 will be incompatible with Python 2.9
    Focus on language + library, not implementation
    Many VMs competing: Jython, IronPython, Parrot..
        Perl has vowed support for their VM
    There will not be a Python 2.10 because Guido hates the ambiguity of the
        
version numbers
    Old-style classes will not be supported.
    
"You have to give the language a chance to evolve, as long as it's slow         
        enough."
--Guido

Improving sum()?
    sum([1,2,3]) returns 6

    class C():
        def __add__(s, o): ...
    sum([C(), C()]) raise an error
    But sum([C(), C()], C()) works
    Why can't sum([C(), C()]) return C() + C()?
    Lots of attempts to "fix" this; all end up flawed
    Conclusion: current behavior is right
    Signature: sum(seq: sequence[T], zero: T = 0) -> T
    IOW, sum() is already perfect!

Adding any() and all()
    def any(seq):
        
for x in seq:
            if x: return True
        return False
    def all(seq):
        for x in seq:
            if not x: return False

        return True
    Why not return x?
        The return type would be surprising if we
fall through the
            loop or if the sequence is empty
.
    Why not anytrue() or anyfalse()?
        if any(x < 0 for x in seq): raise ValueError("input < 0")

        (Note the generator expression above.)

Killing map(), filter(), reduce()
    (This has been on Guido's list for a while -- gathering feedback from the
     community - 50% wildly in favor, 25% opposed, 25% have stated no opinion)
    def map(f, seq):
        return [f(x) for x in seq]

    def filter(p, seq):
        return [x for x in seq if p(x)]

    def reduce(f, seq, zero):
        r = zero
        for x in seq: r = f(r, x)
        return r

    map() is occasionally clearer, e.g. map(str, seq)
    filter() is rarely clearer
    reduce usually more obscure
        often
need to take out pencil & paper (and write out the loop the long
            
way to figure it out)

Killing lambda
    I can smell the next big flame war already :-)

    Example of bad lambda: map(lambda x: x+42, seq)
        - use [x+42 for x in seq]

    But: w.setCallback(lambda: self.setText("Booh"))
    v.  def callback():
            self.setText("Booh")
        w.setCallback(callback)

    Can we add anonymous code blocks
    Lots of people want anonymous blocks - do we really need them?
    w.setCallback({self.setText("Booh")})
        (That was a joke)
    It's far from clear what to do here
        I haven't had time to follow the discussion on py-dev
    Ruby gets a lot of mileage out of anonymous code blocks

        But Ruby doesn't have generators / iterators

    How do you syntactically switch back to multiline syntax when using lambda?

        How do you define how the parser should recognize indentation?
        Is that going to end up being readable for a human

Numeric Needs
    Requirement: 0 dim-array treated as scalar
    Possible solution: where an int is required (e.g. for indexing a sequence),
        and something else is
given, call its __index__ method to convert it to
        an int.

            __index__ only returns an integer if called on something that really
                represents an integer


        Avoids the trap that using __int__ would allow floats to be used as
            indices

    Requirement: control over zero division behavior
    Possible solution: make zero division  a warning, then use the warning
        
module to control its behavior
    
        Return Inf or NaN when no exceptoin raised
        May need to control int and float separately


    Both of these may make it into 2.5 if the numeric people write a PEP

Wake Up!

Optional Static Typing

    NOTE: Nothing's settled yet!!!!!!!!!!!!!!!!
    Strawman syntax:
        def foo(a: int, b: list[int]) -> list[str]: ...

    Strawman semantics:
        def foo(a, b):
            a = __typecheck__(a, int)
            b = ___typecheck__(b, list[int])
            r = __original_foo(a, b)
            return __typecheck__(
r, list[str])

        def __typecheck__(x, T): # new builtin
            if adapt(x, T) is not x: # see PEP 246
                raise TypeError(...)
            return x

        __typecheck__ is overridable per module

    __typecheck__ returns a possibly-converted version of its first argument.

    (class is not a declaration, it's an executable statement -- on the surface it looks a lot like other languages, but deep down, it's totally different)
    
    
Contentious Points

    The syntax
        About 500 people asked whether someone has already proposed using
            'x as int' instead of 'x: int'
        About 50 people suggested 'int x'
        One person proposed 'int(x)'
        The ':' is redundant; we could just write
            def foo(a int, b list[int]) list[str]: ...
                Nobody has pointed this out yet
:-)
    What does list[int] mean, anyway?
        hold that thought...
    And that's not all...


    "You have to be pretty smart and not propose something that comes from Visual Basic" - re the syntax

Interfaces
    I am assuming that this will be introduced together with a syntax for
        interfaces (for example PEP
245)
    Where in the example I wrote list[int] in practice you might be writing
        
sequence[integer] or even iterable[integer] (but not iterator[integer])
    Or perhaps list[int] could actually mean the same thing?
    The
/one/ thing we want to avoid at all cost:
        people writing list[int] because they think in terms of concrete
        types, while the right type to use is sequence[integer] or
        iterable[integer]


__typecheck__ is a Compromise
    Note the trick: __typecheck__ passes iff adapt(x, T)
        returns x itself rather than a wrapper
        This allows people to customize type checking by providing identity
        adapters

    You can override this to use stricter or less strict type checking per
        module by defining a module-global function named __typecheck__


Generic types
    So what does list[int] mean?
    It's a list of ints, duh!

    How does this work in general?
    It could be Python's syntax for generic types
    dict[str, int] is a dict with str keys and int values
    list and dict still work too (same as list[object] etc)
    You can define your own parameteterized types
    Contentious strawman syntax:

        interface List[T](object):
            def append(x: T): "yadda yadda"
            def extend(a: sequence[T]): "..."
            def __getslice__(i: int, j: int) -> List[T]: "..."

How to Typecheck Generic Types
    Ai, that's the rub!
    def foo(x: list[int]) -> int:
        bar(x)
        return x[-1]

    def bar(x: list): # untyped
        x.append(snarf())
    It would be bad enough to have to verify that every item of x is an int upon
        entry into foo()
.
    
It would be even worse to have to do that again after the call to bar(),
        
which might violate the constraint.
    One option: don't check; it's documentation only.

A Psychological Concern
    What if, despite our best efforts, it becomes a style requirement to use
        type declarations everywhere?
    Then we'd have lost!
    It could happen, given the strong meme that, while dynamic typing is
        convenient in the short run, static typing is somehow "better".

    Possible counter-measures:
        dynamic signature checking slows down your code
        add tools to do type inferencing without declarations

        only support type declarations in interfaces
        forget the whole thing
    Type checking is going to slow down your code
.
    Maybe -
O option would cancel type checking--then you'd get the worst of both
        worlds.


Reminder and Disclaimer
    Nothing's settled yet!!!!!
    
Q & A


Q:  
Maps are often faster than list comps. Since you want to kill off map(),
    
what will you do about that?
A:  
Make list comps faster, of course! :-)

Q:  I think that the biggest issue facing python is the "web framework problem"
A:  I think thats that's not the biggest issue, because there's more than the
    web.
    I would like to weigh in, but I may not be the right person to weigh in.
    Maybe having information on the Website would be a good idea

Q:  I would suggest 'strict' typing instead of 'static' typing.
A:  I would agree that the name has to change.


--------------------------------------------------------------------------
REFERENCES: {as documents / sites are referenced add them below}
http://www.elementalsecurity.com
http://www.infoworld.com/article/04/09/24/39FErrdev_1.html
http://www.burtongroup.com

--------------------------------------------------------------------------
QUOTES:
"Perl isn't all bad." - Guido van Rossum
"My favorite 'classic' programming language is Pascal." - Guido van Rossum
"That's why we don't have continuations and we never will. I mean that." - Guido
    van Rossum

--------------------------------------------------------------------------
CONTRIBUTORS: {add your name, e-mail address and URL below}
Ted Leung <twl@sauria.com> <http://www.sauria.com/blog>
Linden Wright <lwright@mac.com>
Nate Silva <nate@northwe.st>

Erik Rose <corp@grinchcentral.com>

Andy Gross <andy@andygross.org>
--------------------------------------------------------------------------
E-MAIL BOUNCEBACK: {add your e-mail address separated by commas }



--------------------------------------------------------------------------
NOTES ON / KEY TO THIS TEMPLATE:
A headline (like a field in a database) will be CAPITALISED
    This differentiates from the text that follows
A variable that you can change will be surrounded by _underscores_
    Spaces in variables are also replaced with under_scores
    This allows people to select the whole variable with a simple double-click
A tool-tip is lower case and surrounded by {curly brackets / parentheses}
    These supply helpful contextual information.

--------------------------------------------------------------------------
Copyright shared between all the participants unless otherwise stated...