New stuff in Python 2.2

Easy stuff
Iterators
Generators
classmethod() and staticmethod()
Descriptors
Properties
Type/class unification
__new__
__slots__
Metaclasses
Secret-bonus section: Python 2.3

Easy stuff

import this

int vs. long

> >> import sys
> >> sys.maxint
2147483647
> >> sys.maxint + 1
2147483648L
> >> # No overflow!

True and False
- These will be real bools in 2.3, for now they're just 1 and 0.

Iterators

Iterators are objects for accessing the items of sequence, one element at a time.
- You usually use them in a for loop.

Example: Files are now iterators

for line in open('/etc/passwd'):
# do something with line

New builtin, iter

> >> iter(open('/etc/passwd'))

Using Iterators

You can also use iterators directly, without a for loop:

> >> i = iter(range(3))
> >> i

> >> i.next()
0
> >> i.next()
1
> >> i.next()
2
> >> i.next()
Traceback (most recent call last):
File " ", line 1, in ?
StopIteration
> >>

Writing your own Iterator

Implement __iter__ on your class

It should return an object with a .next() method

Raise StopIteration when done (if ever)

Generators make this easy...

Generators

These are my favourite addition to 2.2

Generators are resumable functions

Generators implement the Iterator protocol

Example from PEP 255:

from __future__ import generators
def fib():
a, b = 0, 1
while 1:
yield b
a, b = b, a+b

fib in action

Example

> >> g = fib()
> >> g

> >> g.next()
1
> >> g.next()
1
> >> g.next()
2
> >> g.next()
3
> >> g.next()
5
> >> g.next()
8
> >> g.next()
13

fib in action again

Another example

> >> g = fib()
> >> for n, f in zip(range(10), g): print n, f
...
0 1
1 1
2 2
3 3
4 5
5 8
6 13
7 21
8 34
9 55

Generators explained

Any function with a "yield" statement automagically is a generator function

Calling a generator function creates an instance of that generator -- but nothing happens yet

Calling .next() on the resulting generator runs the function as normal
- If the generator returns or raises StopIteration, it finishes
- If it yields a value, it returns that value to the caller of .next(), but remembers where it stopped, and what all the local variables were

Calling .next() again continues from where it stopped...

Generators continued

Because they preserve their state between calls, you can many different generators instatiated at once

They tend to be a very convenient way to implement Iterators: __iter__ can simply return a generator.

staticmethod

Like static methods in C++ and Java

Example:

> >> class C:
... def foo(x, y): # No self!
... print 'static method foo:', x, y
... foo = staticmethod(foo)
...
> >> C.foo(1,2)
static method foo: 1 2
> >> c = C()
> >> c.foo(1,2)
static method foo: 1 2

classmethod

Sort of inbetween static methods and normal methods

Example:

> >> class C:
... def foo(cls, x, y):
... print 'class method foo:', cls, x, y
... foo = classmethod(foo)
...
> >> C.foo(1,2)
class method foo: __main__.C 1 2
> >> c = C()
> >> c.foo(1,2)
class method foo: __main__.C 1 2
> >> class D(C): pass
...
> >> D.foo(1,2)
class method foo: __main__.D 1 2

How does staticmethod work?

Here's a pure Python implementation:

> >> def smethod(x):
... class staticmethod(object):
... def __init__(self, x):
... self.x = x
... def __get__(self, obj, class_=None):
... # Always return an unmolested object
... return self.x
... return staticmethod(x)
...
> >> # Test it
> >> class C:
... def f(x,y): print 'f', x, y
... f = smethod(f)
...
> >> C.f(1,2)
f 1 2
> >> C().f(1,2)
f 1 2

So what the hell is __get__?!

I'm glad you asked ;)

This is the same mechanism that magically:
- makes functions in classes into unbound methods,
- and functions in instances into bound methods.

Part of the "Descriptor" protocol

Descriptors continued

This:

instance.attribute

Actually calls this:

attribute.__get__(instance, instance.__class__)

And this:

MyClass.attribute

Actually calls this:

attribute.__get__(None, MyClass)

There's also __set__ and __delete__.

Properties

Properties are a nice way create attributes that look like attributes, but actually call functions when used.

Example:

class C(object):
def __init__(self):
self.__x = 0

def getx(self):
return self.__x

def setx(self, x):
if x < 0: x = 0
self.__x = x

x = property(getx, setx)

Properties continued

Using the example:

> >> a = C()
> >> a.x = 10
> >> print a.x
10
> >> a.x = -10
> >> print a.x
0
> >> a.setx(12)
> >> print a.getx()
12
> >>

Full prototype is:
- property(fget=None, fset=None, fdel=None, doc=None)

Doesn't work for classic classes

But hang on a second!

What's a "classic" class?

And what is a "new-style" class?

And why have some of the examples had code like:

class Foo(object):
# What the hell is "object"?!

I'm glad you asked! :)

Type/Class unification

In Python 2.2, types, (things like int, str and dict) are now more like classes.

You can subclass from the builtin types!
- More elegant: No more need for UserDict and UserList modules.
- And faster!

Guido says: "This is one of the biggest changes to Python ever"

Subclassing builting types

Example: a simple dict subclass that provides a "default value" that is returned when a missing key is requested

class defaultdict(dict):
def __init__(self, default=None):
dict.__init__(self)
self.default = default

def __getitem__(self, key):
try:
return dict.__getitem__(self, key)
except KeyError:
return self.default

Builtin types are classes

Existing builtins str, int, float, long, complex, list and tuple are now effectively classes
- dict and file have been added to the builtins

Just like a class, you call them to construct an object of that type
- this is completely backwards compatible with how it used to work

Pleasant side-effect: you can now do

> >> l = [1,2,3]
> >> if type(l) is list: print 'yep'
...
yep
> >> # No need to use types.ListType!

object and type

New builtin: object

class C(object):
"""I am a new-style class,
because I inherit from object"""

New builtin: type

> >> object, type, int
( , , )
> >> object.__bases__, type.__bases__, int.__bases__
((), ( ,), ( ,))
> >> type(object), type(type), type(int)
( , , )

Subclassing immutable types

Problem: __init__ is called *after* an object is allocated.

immutable types' contents are already set in stone by __init__

> >> class SquaringTuple(tuple):
... def __init__(self, *args):
... tuple.__init__(self, [x**2 for x in args])
...
> >> t = SquaringTuple(1, 2, 3)
Traceback (most recent call last):
File " ", line 1, in ?
TypeError: tuple() takes at most 1 argument (3 given)

__new__

Solution: __new__, which is called to allocate the object

> >> class SquaringTuple(tuple):
... def __new__(cls, *args):
... return tuple.__new__(cls, [x**2 for x in args])
...
> >> t = SquaringTuple(1, 2, 3)
> >> t
(1, 4, 9)

__new__ is called with a class as its first argument; its responsibility is to return a new instance of that class

Compare this to __init__: __init__ is called with an instance as its first argument, and it doesn't return anything; its responsibility is to initialise the instance

__new__ continued

__new__ could return a cached instance of a class

__new__ can return an instance of a completely different class!
- But you need Python 2.2.2 if you want the right __init__ to be called

__slots__

Normally, every object instance has a __dict__ attribute -- a complete dictionary object.

This is often unnecessary -- e.g when subclassing dict!
- The defaultdict class from earlier would take twice the memory of a plain dict

Solution: You can define a __slots__ attribute on new-style classes
- it reserves space for the attributes named in it, and only those attributes
- no __dict__ is allocated

__slots__ continued

Example

> >> class C(object):
... __slots__ = ['x', 'y']
...
> >> c = C()
> >> c.x = 1
> >> c.x
1
> >> c.z = 2
Traceback (most recent call last):
File " ", line 1, in ?
AttributeError: 'C' object has no attribute 'z'

Metaclasses

"In the past, the subject of metaclasses in Python has caused hairs to raise and even brains to explode"

"Fortunately, in Python 2.2, metaclasses are more accessible and less dangerous"

This is going to be a really brief peek at metaclasses. There just isn't enough time to discuss them properly in this talk.

What defines a class?
- Name, base classes, and its namespace (a dict)

Metaclasses in 1 easy slide

> >> class capitalise(type):
... def __init__(cls, name, bases, dict):
... type.__init__(cls, name, bases, dict)
... for name in dict:
... NAME = name.upper()
... if NAME not in dict:
... setattr(cls, NAME, dict[name])
...
> >> class C:
... __metaclass__ = capitalise
... def foo(self):
... print 'foo'
...
> >> c = C()
> >> c.foo()
foo
> >> c.FOO()
foo

Secret-Ultra-Mega Bonus: Python 2.3

enumerate

> >> s = "Hello from Guido's time machine!'
> >> for index, word in enumerate(s.split()):
... print index, word
...
0 Hello
1 from
2 Guido's
3 time
4 machine!

faster list sorting

A standard Set module
- This may eventually become a builtin

Python 2.3 continued

Extended slices

> >> range(10)[::-2]
[9, 7, 5, 3, 1]

A standard logging module

Universal Newline support
- can automatically convert all line endings into \n

Lots of little optimisations
- Benchmarks suggest that 2.3 is about 25% faster!

And lots more!
- Not as insanely action-packed as 2.2, though ;)