SVNROOT = http://svn.python.org/projects
SPHINXOPTS =
PAPER =
+SOURCES =
ALLSPHINXOPTS = -b $(BUILDER) -d build/doctrees -D latex_paper_size=$(PAPER) \
- $(SPHINXOPTS) . build/$(BUILDER)
+ $(SPHINXOPTS) . build/$(BUILDER) $(SOURCES)
.PHONY: help checkout update build html web htmlhelp clean coverage
coverage: build
@echo "Coverage finished; see c.txt and python.txt in build/coverage"
+doctest: BUILDER = doctest
+doctest: build
+ @echo "Testing of doctests in the sources finished, look at the " \
+ "results in build/doctest/output.txt"
+
clean:
-rm -rf build/*
-rm -rf tools/sphinx
# General configuration
# ---------------------
-extensions = ['sphinx.ext.refcounting', 'sphinx.ext.coverage']
+extensions = ['sphinx.ext.refcounting', 'sphinx.ext.coverage',
+ 'sphinx.ext.doctest']
templates_path = ['tools/sphinxext']
# General substitutions.
Functional Programming HOWTO
********************************
-:Author: \A. M. Kuchling
+:Author: A. M. Kuchling
:Release: 0.31
(This is a first draft. Please send comments/error reports/suggestions to
* Composability.
* Ease of debugging and testing.
+
Formal provability
------------------
proof; maybe there's an error in it, and you wrongly believe you've proved the
program correct.
+
Modularity
----------
check that the output matches expectations.
-
Composability
-------------
a few functions specialized for the current task.
-
Iterators
=========
dictionaries. An object is called an **iterable** object if you can get an
iterator for it.
-You can experiment with the iteration interface manually::
+You can experiment with the iteration interface manually:
>>> L = [1,2,3]
>>> it = iter(L)
>>> it
- <iterator object at 0x8116870>
+ <...iterator object at ...>
>>> it.next()
1
>>> it.next()
be an iterator or some object for which ``iter()`` can create an iterator.
These two statements are equivalent::
- for i in iter(obj):
- print(i)
- for i in obj:
- print(i)
+ for i in iter(obj):
+ print i
+
+ for i in obj:
+ print i
Iterators can be materialized as lists or tuples by using the :func:`list` or
-:func:`tuple` constructor functions::
+:func:`tuple` constructor functions:
>>> L = [1,2,3]
>>> iterator = iter(L)
(1, 2, 3)
Sequence unpacking also supports iterators: if you know an iterator will return
-N elements, you can unpack them into an N-tuple::
+N elements, you can unpack them into an N-tuple:
>>> L = [1,2,3]
>>> iterator = iter(L)
iterator.
Calling :func:`iter` on a dictionary returns an iterator that will loop over the
-dictionary's keys::
+dictionary's keys:
+
+.. not a doctest since dict ordering varies across Pythons
+
+::
>>> m = {'Jan': 1, 'Feb': 2, 'Mar': 3, 'Apr': 4, 'May': 5, 'Jun': 6,
... 'Jul': 7, 'Aug': 8, 'Sep': 9, 'Oct': 10, 'Nov': 11, 'Dec': 12}
Feb 2
Aug 8
Sep 9
- May 5
+ Apr 4
Jun 6
Jul 7
Jan 1
- Apr 4
+ May 5
Nov 11
Dec 12
Oct 10
:meth:`values` or :meth:`items` methods to get an appropriate iterator.
The :func:`dict` constructor can accept an iterator that returns a finite stream
-of ``(key, value)`` tuples::
+of ``(key, value)`` tuples:
>>> L = [('Italy', 'Rome'), ('France', 'Paris'), ('US', 'Washington DC')]
>>> dict(iter(L))
functional programming language Haskell (http://www.haskell.org). You can strip
all the whitespace from a stream of strings with the following code::
- line_list = [' line 1\n', 'line 2 \n', ...]
+ line_list = [' line 1\n', 'line 2 \n', ...]
- # Generator expression -- returns iterator
- stripped_iter = (line.strip() for line in line_list)
+ # Generator expression -- returns iterator
+ stripped_iter = (line.strip() for line in line_list)
- # List comprehension -- returns list
- stripped_list = [line.strip() for line in line_list]
+ # List comprehension -- returns list
+ stripped_list = [line.strip() for line in line_list]
You can select only certain elements by adding an ``"if"`` condition::
- stripped_list = [line.strip() for line in line_list
- if line != ""]
+ stripped_list = [line.strip() for line in line_list
+ if line != ""]
With a list comprehension, you get back a Python list; ``stripped_list`` is a
list containing the resulting lines, not an iterator. Generator expressions
parentheses signalling a function call also count. If you want to create an
iterator that will be immediately passed to a function you can write::
- obj_total = sum(obj.count for obj in list_all_objects())
+ obj_total = sum(obj.count for obj in list_all_objects())
The ``for...in`` clauses contain the sequences to be iterated over. The
sequences do not have to be the same length, because they are iterated over from
This means that when there are multiple ``for...in`` clauses but no ``if``
clauses, the length of the resulting output will be equal to the product of the
lengths of all the sequences. If you have two lists of length 3, the output
-list is 9 elements long::
+list is 9 elements long:
- seq1 = 'abc'
- seq2 = (1,2,3)
- >>> [ (x,y) for x in seq1 for y in seq2]
+.. doctest::
+ :options: +NORMALIZE_WHITESPACE
+
+ >>> seq1 = 'abc'
+ >>> seq2 = (1,2,3)
+ >>> [(x,y) for x in seq1 for y in seq2]
[('a', 1), ('a', 2), ('a', 3),
('b', 1), ('b', 2), ('b', 3),
('c', 1), ('c', 2), ('c', 3)]
function? What if you could later resume the function where it left off? This
is what generators provide; they can be thought of as resumable functions.
-Here's the simplest example of a generator function::
+Here's the simplest example of a generator function:
+
+.. testcode::
def generate_ints(N):
for i in range(N):
suspended and local variables are preserved. On the next call to the
generator's ``.next()`` method, the function will resume executing.
-Here's a sample usage of the ``generate_ints()`` generator::
+Here's a sample usage of the ``generate_ints()`` generator:
>>> gen = generate_ints(3)
>>> gen
- <generator object at 0x8117f90>
+ <generator object at ...>
>>> gen.next()
0
>>> gen.next()
The test suite included with Python's library, ``test_generators.py``, contains
a number of more interesting examples. Here's one generator that implements an
-in-order traversal of a tree using generators recursively.
-
-::
+in-order traversal of a tree using generators recursively. ::
# A recursive generator that generates Tree leaves in in-order.
def inorder(t):
Here's a simple counter that increments by 1 and allows changing the value of
the internal counter.
-::
+.. testcode::
def counter (maximum):
i = 0
``map(f, iterA, iterB, ...)`` returns an iterator over the sequence
``f(iterA[0], iterB[0]), f(iterA[1], iterB[1]), f(iterA[2], iterB[2]), ...``.
-::
+ >>> def upper(s):
+ ... return s.upper()
- def upper(s):
- return s.upper()
- list(map(upper, ['sentence', 'fragment'])) =>
- ['SENTENCE', 'FRAGMENT']
- list(upper(s) for s in ['sentence', 'fragment']) =>
- ['SENTENCE', 'FRAGMENT']
+ >>> map(upper, ['sentence', 'fragment'])
+ ['SENTENCE', 'FRAGMENT']
+ >>> [upper(s) for s in ['sentence', 'fragment']]
+ ['SENTENCE', 'FRAGMENT']
You can of course achieve the same effect with a list comprehension.
some condition; for use with :func:`filter`, the predicate must take a single
value.
-::
+ >>> def is_even(x):
+ ... return (x % 2) == 0
- def is_even(x):
- return (x % 2) == 0
+ >>> filter(is_even, range(10))
+ [0, 2, 4, 6, 8]
- list(filter(is_even, range(10))) =>
- [0, 2, 4, 6, 8]
-This can also be written as a generator expression::
+This can also be written as a list comprehension:
>>> list(x for x in range(10) if is_even(x))
[0, 2, 4, 6, 8]
raised. If the initial value is supplied, it's used as a starting point and
``func(initial_value, A)`` is the first calculation. ::
- import operator
- import functools
- functools.reduce(operator.concat, ['A', 'BB', 'C']) =>
- 'ABBC'
- functools.reduce(operator.concat, []) =>
- TypeError: reduce() of empty sequence with no initial value
- functools.reduce(operator.mul, [1,2,3], 1) =>
- 6
- functools.reduce(operator.mul, [], 1) =>
- 1
-
-If you use :func:`operator.add` with :func:`functools.reduce`, you'll add up all
-the elements of the iterable. This case is so common that there's a special
-built-in called :func:`sum` to compute it::
-
- functools.reduce(operator.add, [1,2,3,4], 0) =>
- 10
- sum([1,2,3,4]) =>
- 10
- sum([]) =>
- 0
+
+``reduce(func, iter, [initial_value])`` doesn't have a counterpart in the
+:mod:`itertools` module because it cumulatively performs an operation on all the
+iterable's elements and therefore can't be applied to infinite iterables.
+``func`` must be a function that takes two elements and returns a single value.
+:func:`reduce` takes the first two elements A and B returned by the iterator and
+calculates ``func(A, B)``. It then requests the third element, C, calculates
+``func(func(A, B), C)``, combines this result with the fourth element returned,
+and continues until the iterable is exhausted. If the iterable returns no
+values at all, a :exc:`TypeError` exception is raised. If the initial value is
+supplied, it's used as a starting point and ``func(initial_value, A)`` is the
+first calculation.
+
+ >>> import operator
+ >>> reduce(operator.concat, ['A', 'BB', 'C'])
+ 'ABBC'
+ >>> reduce(operator.concat, [])
+ Traceback (most recent call last):
+ ...
+ TypeError: reduce() of empty sequence with no initial value
+ >>> reduce(operator.mul, [1,2,3], 1)
+ 6
+ >>> reduce(operator.mul, [], 1)
+ 1
+
+If you use :func:`operator.add` with :func:`reduce`, you'll add up all the
+elements of the iterable. This case is so common that there's a special
+built-in called :func:`sum` to compute it:
+
+ >>> reduce(operator.add, [1,2,3,4], 0)
+ 10
+ >>> sum([1,2,3,4])
+ 10
+ >>> sum([])
+ 0
For many uses of :func:`reduce`, though, it can be clearer to just write the
obvious :keyword:`for` loop::
``enumerate(iter)`` counts off the elements in the iterable, returning 2-tuples
containing the count and each element. ::
- enumerate(['subject', 'verb', 'object']) =>
- (0, 'subject'), (1, 'verb'), (2, 'object')
+ >>> for item in enumerate(['subject', 'verb', 'object']):
+ ... print item
+ (0, 'subject')
+ (1, 'verb')
+ (2, 'object')
:func:`enumerate` is often used when looping through a list and recording the
indexes at which certain conditions are met::
if line.strip() == '':
print('Blank line at line #%i' % i)
-``sorted(iterable, [key=None], [reverse=False)`` collects all the elements of
-the iterable into a list, sorts the list, and returns the sorted result. The
-``key``, and ``reverse`` arguments are passed through to the constructed list's
-``sort()`` method. ::
-
- import random
- # Generate 8 random numbers between [0, 10000)
- rand_list = random.sample(range(10000), 8)
- rand_list =>
- [769, 7953, 9828, 6431, 8442, 9878, 6213, 2207]
- sorted(rand_list) =>
- [769, 2207, 6213, 6431, 7953, 8442, 9828, 9878]
- sorted(rand_list, reverse=True) =>
- [9878, 9828, 8442, 7953, 6431, 6213, 2207, 769]
+
+``sorted(iterable, [cmp=None], [key=None], [reverse=False)`` collects all the
+elements of the iterable into a list, sorts the list, and returns the sorted
+result. The ``cmp``, ``key``, and ``reverse`` arguments are passed through to
+the constructed list's ``.sort()`` method. ::
+
+ >>> import random
+ >>> # Generate 8 random numbers between [0, 10000)
+ >>> rand_list = random.sample(range(10000), 8)
+ >>> rand_list
+ [769, 7953, 9828, 6431, 8442, 9878, 6213, 2207]
+ >>> sorted(rand_list)
+ [769, 2207, 6213, 6431, 7953, 8442, 9828, 9878]
+ >>> sorted(rand_list, reverse=True)
+ [9878, 9828, 8442, 7953, 6431, 6213, 2207, 769]
(For a more detailed discussion of sorting, see the Sorting mini-HOWTO in the
Python wiki at http://wiki.python.org/moin/HowTo/Sorting.)
The ``any(iter)`` and ``all(iter)`` built-ins look at the truth values of an
iterable's contents. :func:`any` returns True if any element in the iterable is
a true value, and :func:`all` returns True if all of the elements are true
-values::
-
- any([0,1,0]) =>
- True
- any([0,0,0]) =>
- False
- any([1,1,1]) =>
- True
- all([0,1,0]) =>
- False
- all([0,0,0]) =>
- False
- all([1,1,1]) =>
- True
+values:
+
+ >>> any([0,1,0])
+ True
+ >>> any([0,0,0])
+ False
+ >>> any([1,1,1])
+ True
+ >>> all([0,1,0])
+ False
+ >>> all([0,0,0])
+ False
+ >>> all([1,1,1])
+ True
Small functions and the lambda expression
If there's a Python built-in or a module function that's suitable, you don't
need to define a new function at all::
- stripped_lines = [line.strip() for line in lines]
- existing_files = filter(os.path.exists, file_list)
+ stripped_lines = [line.strip() for line in lines]
+ existing_files = filter(os.path.exists, file_list)
If the function you need doesn't exist, you need to write it. One way to write
small functions is to use the ``lambda`` statement. ``lambda`` takes a number
of parameters and an expression combining these parameters, and creates a small
function that returns the value of the expression::
- lowercase = lambda x: x.lower()
+ lowercase = lambda x: x.lower()
- print_assign = lambda name, value: name + '=' + str(value)
+ print_assign = lambda name, value: name + '=' + str(value)
- adder = lambda x, y: x+y
+ adder = lambda x, y: x+y
An alternative is to just use the ``def`` statement and define a function in the
usual way::
- def lowercase(x):
- return x.lower()
+ def lowercase(x):
+ return x.lower()
- def print_assign(name, value):
- return name + '=' + str(value)
+ def print_assign(name, value):
+ return name + '=' + str(value)
- def adder(x,y):
- return x + y
+ def adder(x,y):
+ return x + y
Which alternative is preferable? That's a style question; my usual course is to
avoid using ``lambda``.
``itertools.count(n)`` returns an infinite stream of integers, increasing by 1
each time. You can optionally supply the starting number, which defaults to 0::
- itertools.count() =>
- 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, ...
- itertools.count(10) =>
- 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, ...
+ itertools.count() =>
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, ...
+ itertools.count(10) =>
+ 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, ...
``itertools.cycle(iter)`` saves a copy of the contents of a provided iterable
and returns a new iterator that returns its elements from first to last. The
-new iterator will repeat these elements infinitely.
-
-::
+new iterator will repeat these elements infinitely. ::
- itertools.cycle([1,2,3,4,5]) =>
- 1, 2, 3, 4, 5, 1, 2, 3, 4, 5, ...
+ itertools.cycle([1,2,3,4,5]) =>
+ 1, 2, 3, 4, 5, 1, 2, 3, 4, 5, ...
``itertools.repeat(elem, [n])`` returns the provided element ``n`` times, or
-returns the element endlessly if ``n`` is not provided.
-
-::
+returns the element endlessly if ``n`` is not provided. ::
itertools.repeat('abc') =>
abc, abc, abc, abc, abc, abc, abc, abc, abc, abc, ...
``itertools.chain(iterA, iterB, ...)`` takes an arbitrary number of iterables as
input, and returns all the elements of the first iterator, then all the elements
-of the second, and so on, until all of the iterables have been exhausted.
-
-::
+of the second, and so on, until all of the iterables have been exhausted. ::
itertools.chain(['a', 'b', 'c'], (1, 2, 3)) =>
a, b, c, 1, 2, 3
This iterator is intended to be used with iterables that are all of the same
length. If the iterables are of different lengths, the resulting stream will be
-the same length as the shortest iterable.
-
-::
+the same length as the shortest iterable. ::
itertools.izip(['a', 'b'], (1, 2, 3)) =>
('a', 1), ('b', 2)
first ``stop`` elements. If you supply a starting index, you'll get
``stop-start`` elements, and if you supply a value for ``step``, elements will
be skipped accordingly. Unlike Python's string and list slicing, you can't use
-negative values for ``start``, ``stop``, or ``step``.
-
-::
+negative values for ``start``, ``stop``, or ``step``. ::
itertools.islice(range(10), 8) =>
0, 1, 2, 3, 4, 5, 6, 7
If you don't supply a value for ``n``, the default is 2. Replicating iterators
requires saving some of the contents of the source iterator, so this can consume
significant memory if the iterator is large and one of the new iterators is
-consumed more than the others.
-
-::
+consumed more than the others. ::
itertools.tee( itertools.count() ) =>
iterA, iterB
The ``compose()`` function implements function composition. In other words, it
returns a wrapper around the ``outer`` and ``inner`` callables, such that the
-return value from ``inner`` is fed directly to ``outer``. That is,
-
-::
-
- >>> def add(a, b):
- ... return a + b
- ...
- >>> def double(a):
- ... return 2 * a
- ...
- >>> compose(double, add)(5, 6)
- 22
+return value from ``inner`` is fed directly to ``outer``. That is, ::
-is equivalent to
+ >>> def add(a, b):
+ ... return a + b
+ ...
+ >>> def double(a):
+ ... return 2 * a
+ ...
+ >>> compose(double, add)(5, 6)
+ 22
-::
+is equivalent to ::
- >>> double(add(5, 6))
- 22
+ >>> double(add(5, 6))
+ 22
The ``unpack`` keyword is provided to work around the fact that Python functions
are not always `fully curried <http://en.wikipedia.org/wiki/Currying>`__. By
default, it is expected that the ``inner`` function will return a single object
and that the ``outer`` function will take a single argument. Setting the
``unpack`` argument causes ``compose`` to expect a tuple from ``inner`` which
-will be expanded before being passed to ``outer``. Put simply,
-
-::
+will be expanded before being passed to ``outer``. Put simply, ::
- compose(f, g)(5, 6)
+ compose(f, g)(5, 6)
is equivalent to::
- f(g(5, 6))
+ f(g(5, 6))
-while
+while ::
-::
-
- compose(f, g, unpack=True)(5, 6)
+ compose(f, g, unpack=True)(5, 6)
is equivalent to::
- f(*g(5, 6))
+ f(*g(5, 6))
Even though ``compose()`` only accepts two functions, it's trivial to build up a
version that will compose any number of functions. We'll use ``functools.reduce()``,
``compose()`` and ``partial()`` (the last of which is provided by both
-``functional`` and ``functools``).
-
-::
+``functional`` and ``functools``). ::
- from functional import compose, partial
+ from functional import compose, partial
- multi_compose = partial(functools.reduce, compose)
+
+ multi_compose = partial(reduce, compose)
We can also use ``map()``, ``compose()`` and ``partial()`` to craft a version of
``"".join(...)`` that converts its arguments to string::
- from functional import compose, partial
+ from functional import compose, partial
- join = compose("".join, partial(map, str))
+ join = compose("".join, partial(map, str))
``flip(func)``
``flip()`` wraps the callable in ``func`` and causes it to receive its
-non-keyword arguments in reverse order.
-
-::
-
- >>> def triple(a, b, c):
- ... return (a, b, c)
- ...
- >>> triple(5, 6, 7)
- (5, 6, 7)
- >>>
- >>> flipped_triple = flip(triple)
- >>> flipped_triple(5, 6, 7)
- (7, 6, 5)
+non-keyword arguments in reverse order. ::
+
+ >>> def triple(a, b, c):
+ ... return (a, b, c)
+ ...
+ >>> triple(5, 6, 7)
+ (5, 6, 7)
+ >>>
+ >>> flipped_triple = flip(triple)
+ >>> flipped_triple(5, 6, 7)
+ (7, 6, 5)
``foldl(func, start, iterable)``
This means that a call such as::
- foldl(f, 0, [1, 2, 3])
+ foldl(f, 0, [1, 2, 3])
is equivalent to::
- f(f(f(0, 1), 2), 3)
+ f(f(f(0, 1), 2), 3)
``foldl()`` is roughly equivalent to the following recursive function::
- def foldl(func, start, seq):
- if len(seq) == 0:
- return start
+ def foldl(func, start, seq):
+ if len(seq) == 0:
+ return start
- return foldl(func, func(start, seq[0]), seq[1:])
+ return foldl(func, func(start, seq[0]), seq[1:])
Speaking of equivalence, the above ``foldl`` call can be expressed in terms of
the built-in ``reduce`` like so::
- reduce(f, [1, 2, 3], 0)
+ reduce(f, [1, 2, 3], 0)
We can use ``foldl()``, ``operator.concat()`` and ``partial()`` to write a
cleaner, more aesthetically-pleasing version of Python's ``"".join(...)``
idiom::
- from functional import foldl, partial
- from operator import concat
-
- join = partial(foldl, concat, "")
+ from functional import foldl, partial from operator import concat
+
+ join = partial(foldl, concat, "")
Revision History and Acknowledgements
:func:`encodestring` returns a string containing one or more lines of
base64-encoded data always including an extra trailing newline (``'\n'``).
-An example usage of the module::
+An example usage of the module:
>>> import base64
>>> encoded = base64.b64encode('data to be encoded')
The :func:`bisect` function is generally useful for categorizing numeric data.
This example uses :func:`bisect` to look up a letter grade for an exam total
(say) based on a set of ordered numeric breakpoints: 85 and up is an 'A', 75..84
-is a 'B', etc. ::
+is a 'B', etc.
>>> grades = "FEDCBA"
>>> breakpoints = [30, 44, 66, 75, 85]
.. moduleauthor:: Raymond Hettinger <python@rcn.com>
.. sectionauthor:: Raymond Hettinger <python@rcn.com>
+.. testsetup:: *
+
+ from collections import *
+ import itertools
+ __name__ = '<doctest>'
This module implements high-performance container datatypes. Currently,
there are two datatypes, :class:`deque` and :class:`defaultdict`, and
:class:`list` isn't convenient.
The specialized containers provided in this module provide alternatives
-to Python's general purpose built-in containers, :class:`dict`,
+to Python's general purpose built-in containers, :class:`dict`,
:class:`list`, :class:`set`, and :class:`tuple`.
Besides the containers provided here, the optional :mod:`bsddb`
-module offers the ability to create in-memory or file based ordered
+module offers the ability to create in-memory or file based ordered
dictionaries with string keys using the :meth:`bsddb.btopen` method.
In addition to containers, the collections module provides some ABCs
-(abstract base classes). These can be used to test whether a class
+(abstract base classes) that can be used to test whether a class
provides a particular interface, for example, is it hashable or
a mapping, and some of them can also be used as mixin classes.
Notes on using :class:`Set` and :class:`MutableSet` as a mixin:
-(1)
+(1)
Since some set operations create new sets, the default mixin methods need
- a way to create new instances from an iterable. The class constructor is
- assumed to have a signature in the form ``ClassName(iterable)``.
+ a way to create new instances from an iterable. The class constructor is
+ assumed to have a signature in the form ``ClassName(iterable)``.
That assumption is factored-out to a single internal classmethod called
:meth:`_from_iterable` which calls ``cls(iterable)`` to produce a new set.
If the :class:`Set` mixin is being used in a class with a different
- constructor signature, you will need to override :meth:`from_iterable`
- with a classmethod that can construct new instances from
+ constructor signature, you will need to override :meth:`from_iterable`
+ with a classmethod that can construct new instances from
an iterable argument.
(2)
``reversed(d)``, ``copy.copy(d)``, ``copy.deepcopy(d)``, membership testing with
the :keyword:`in` operator, and subscript references such as ``d[-1]``.
-Example::
+Example:
+
+.. doctest::
>>> from collections import deque
>>> d = deque('ghi') # make a new deque with three items
>>> for elem in d: # iterate over the deque's elements
- ... print(elem.upper())
+ ... print elem.upper()
G
H
I
deque.
For example, building a balanced binary tree of nested lists entails reducing
-two adjacent nodes into one by grouping them in a list::
+two adjacent nodes into one by grouping them in a list:
>>> def maketree(iterable):
... d = deque(iterable)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Using :class:`list` as the :attr:`default_factory`, it is easy to group a
-sequence of key-value pairs into a dictionary of lists::
+sequence of key-value pairs into a dictionary of lists:
>>> s = [('yellow', 1), ('blue', 2), ('yellow', 3), ('blue', 4), ('red', 1)]
>>> d = defaultdict(list)
operation then attaches the value to the new list. When keys are encountered
again, the look-up proceeds normally (returning the list for that key) and the
:meth:`list.append` operation adds another value to the list. This technique is
-simpler and faster than an equivalent technique using :meth:`dict.setdefault`::
+simpler and faster than an equivalent technique using :meth:`dict.setdefault`:
>>> d = {}
>>> for k, v in s:
Setting the :attr:`default_factory` to :class:`int` makes the
:class:`defaultdict` useful for counting (like a bag or multiset in other
-languages)::
+languages):
>>> s = 'mississippi'
>>> d = defaultdict(int)
The function :func:`int` which always returns zero is just a special case of
constant functions. A faster and more flexible way to create constant functions
is to use a lambda function which can supply any constant value (not just
-zero)::
+zero):
>>> def constant_factory(value):
... return lambda: value
'John ran to <missing>'
Setting the :attr:`default_factory` to :class:`set` makes the
-:class:`defaultdict` useful for building a dictionary of sets::
+:class:`defaultdict` useful for building a dictionary of sets:
>>> s = [('red', 1), ('blue', 2), ('red', 3), ('blue', 4), ('red', 1), ('blue', 4)]
>>> d = defaultdict(set)
Named tuple instances do not have per-instance dictionaries, so they are
lightweight and require no more memory than regular tuples.
-Example::
+Example:
+
+.. doctest::
+ :options: +NORMALIZE_WHITESPACE
>>> Point = namedtuple('Point', 'x y', verbose=True)
class Point(tuple):
'Point(x, y)'
-
+ <BLANKLINE>
__slots__ = ()
-
+ <BLANKLINE>
_fields = ('x', 'y')
-
+ <BLANKLINE>
def __new__(cls, x, y):
return tuple.__new__(cls, (x, y))
-
+ <BLANKLINE>
@classmethod
- def _make(cls, iterable):
+ def _make(cls, iterable, new=tuple.__new__, len=len):
'Make a new Point object from a sequence or iterable'
- result = tuple.__new__(cls, iterable)
+ result = new(cls, iterable)
if len(result) != 2:
raise TypeError('Expected 2 arguments, got %d' % len(result))
return result
-
+ <BLANKLINE>
def __repr__(self):
return 'Point(x=%r, y=%r)' % self
-
+ <BLANKLINE>
def _asdict(t):
'Return a new dict which maps field names to their values'
return {'x': t[0], 'y': t[1]}
-
+ <BLANKLINE>
def _replace(self, **kwds):
'Return a new Point object replacing specified fields with new values'
result = self._make(map(kwds.pop, ('x', 'y'), self))
if kwds:
raise ValueError('Got unexpected field names: %r' % kwds.keys())
return result
-
+ <BLANKLINE>
x = property(itemgetter(0))
y = property(itemgetter(1))
Class method that makes a new instance from an existing sequence or iterable.
-::
+.. doctest::
>>> t = [11, 22]
>>> Point._make(t)
.. method:: somenamedtuple._asdict()
- Return a new dict which maps field names to their corresponding values:
-
-::
+ Return a new dict which maps field names to their corresponding values::
>>> p._asdict()
{'x': 11, 'y': 22}
-
+
.. method:: somenamedtuple._replace(kwargs)
- Return a new instance of the named tuple replacing specified fields with new values:
+ Return a new instance of the named tuple replacing specified fields with new
+ values:
::
Tuple of strings listing the field names. Useful for introspection
and for creating new named tuple types from existing named tuples.
-::
+.. doctest::
>>> p._fields # view the field names
('x', 'y')
Pixel(x=11, y=22, red=128, green=255, blue=0)
To retrieve a field whose name is stored in a string, use the :func:`getattr`
-function::
+function:
>>> getattr(p, 'x')
11
-To convert a dictionary to a named tuple, use the double-star-operator [#]_::
+To convert a dictionary to a named tuple, use the double-star-operator [#]_:
>>> d = {'x': 11, 'y': 22}
>>> Point(**d)
Since a named tuple is a regular Python class, it is easy to add or change
functionality with a subclass. Here is how to add a calculated field and
-a fixed-width print format::
+a fixed-width print format:
>>> class Point(namedtuple('Point', 'x y')):
... __slots__ = ()
>>> for p in Point(3, 4), Point(14, 5/7.):
... print(p)
-
Point: x= 3.000 y= 4.000 hypot= 5.000
Point: x=14.000 y= 0.714 hypot=14.018
Subclassing is not useful for adding new, stored fields. Instead, simply
-create a new named tuple type from the :attr:`_fields` attribute::
+create a new named tuple type from the :attr:`_fields` attribute:
>>> Point3D = namedtuple('Point3D', Point._fields + ('z',))
Default values can be implemented by using :meth:`_replace` to
-customize a prototype instance::
+customize a prototype instance:
>>> Account = namedtuple('Account', 'owner balance transaction_count')
>>> default_account = Account('<owner name>', 0.0, 0)
Example
-------
-The following example demonstrates how to use the :mod:`Cookie` module. ::
+The following example demonstrates how to use the :mod:`Cookie` module.
+
+.. doctest::
+ :options: +NORMALIZE_WHITESPACE
>>> import Cookie
>>> C = Cookie.SimpleCookie()
>>> C["fig"] = "newton"
>>> C["sugar"] = "wafer"
>>> print(C) # generate HTTP headers
- Set-Cookie: sugar=wafer
Set-Cookie: fig=newton
- >>> print(C.output()) # same thing
Set-Cookie: sugar=wafer
+ >>> print(C.output()) # same thing
Set-Cookie: fig=newton
+ Set-Cookie: sugar=wafer
>>> C = Cookie.SmartCookie()
>>> C["rocky"] = "road"
>>> C["rocky"]["path"] = "/cookie"
>>> C = Cookie.SmartCookie()
>>> C.load("chips=ahoy; vienna=finger") # load from a string (HTTP header)
>>> print(C)
- Set-Cookie: vienna=finger
Set-Cookie: chips=ahoy
+ Set-Cookie: vienna=finger
>>> C = Cookie.SmartCookie()
>>> C.load('keebler="E=everybody; L=\\"Loves\\"; fudge=\\012;";')
>>> print(C)
:exc:`OverflowError` is raised.
Note that normalization of negative values may be surprising at first. For
- example, ::
+ example,
>>> from datetime import timedelta
>>> d = timedelta(microseconds=-1)
efficient pickling, and in Boolean contexts, a :class:`timedelta` object is
considered to be true if and only if it isn't equal to ``timedelta(0)``.
-Example usage::
+Example usage:
>>> from datetime import timedelta
>>> year = timedelta(days=365)
>>> time_to_birthday.days
202
-Example of working with :class:`date`::
+Example of working with :class:`date`:
+
+.. doctest::
>>> from datetime import date
>>> d = date.fromordinal(730920) # 730920th day after 1. 1. 0001
>>> d
datetime.date(2002, 3, 11)
>>> t = d.timetuple()
- >>> for i in t:
+ >>> for i in t: # doctest: +SKIP
... print i
2002 # year
3 # month
70 # 70th day in the year
-1
>>> ic = d.isocalendar()
- >>> for i in ic:
- ... print i # doctest: +SKIP
+ >>> for i in ic: # doctest: +SKIP
+ ... print i
2002 # ISO year
11 # ISO week number
1 # ISO day number ( 1 = Monday )
YYYY-MM-DDTHH:MM:SS+HH:MM
The optional argument *sep* (default ``'T'``) is a one-character separator,
- placed between the date and time portions of the result. For example, ::
+ placed between the date and time portions of the result. For example,
>>> from datetime import tzinfo, timedelta, datetime
>>> class TZ(tzinfo):
Return a string representing the date and time, controlled by an explicit format
string. See section :ref:`strftime-behavior`.
-Examples of working with datetime objects::
-
+Examples of working with datetime objects:
+
+.. doctest::
+
>>> from datetime import datetime, date, time
>>> # Using datetime.combine()
>>> d = date(2005, 7, 14)
>>> datetime.combine(d, t)
datetime.datetime(2005, 7, 14, 12, 30)
>>> # Using datetime.now() or datetime.utcnow()
- >>> datetime.now()
+ >>> datetime.now() # doctest: +SKIP
datetime.datetime(2007, 12, 6, 16, 29, 43, 79043) # GMT +1
- >>> datetime.utcnow()
+ >>> datetime.utcnow() # doctest: +SKIP
datetime.datetime(2007, 12, 6, 15, 29, 43, 79060)
>>> # Using datetime.strptime()
>>> dt = datetime.strptime("21/11/06 16:30", "%d/%m/%y %H:%M")
datetime.datetime(2006, 11, 21, 16, 30)
>>> # Using datetime.timetuple() to get tuple of all attributes
>>> tt = dt.timetuple()
- >>> for it in tt:
+ >>> for it in tt: # doctest: +SKIP
... print it
...
2006 # year
-1 # dst - method tzinfo.dst() returned None
>>> # Date in ISO format
>>> ic = dt.isocalendar()
- >>> for it in ic:
+ >>> for it in ic: # doctest: +SKIP
... print it
...
2006 # ISO year
>>> dt.strftime("%A, %d. %B %Y %I:%M%p")
'Tuesday, 21. November 2006 04:30PM'
-Using datetime with tzinfo::
+Using datetime with tzinfo:
>>> from datetime import timedelta, datetime, tzinfo
>>> class GMT1(tzinfo):
``self.tzinfo.tzname(None)``, or raises an exception if the latter doesn't
return ``None`` or a string object.
-Example::
+Example:
>>> from datetime import time, tzinfo
>>> class GMT1(tzinfo):
.. moduleauthor:: Tim Peters <tim.one at comcast.net>
.. sectionauthor:: Raymond D. Hettinger <python at rcn.com>
+.. import modules for testing inline doctests with the Sphinx doctest builder
+.. testsetup:: *
+
+ import decimal
+ import math
+ from decimal import *
+ # make sure each group gets a fresh context
+ setcontext(Context())
The :mod:`decimal` module provides support for decimal floating point
arithmetic. It offers several advantages over the :class:`float` datatype:
* Unlike hardware based binary floating point, the decimal module has a user
alterable precision (defaulting to 28 places) which can be as large as needed for
- a given problem::
+ a given problem:
>>> getcontext().prec = 6
>>> Decimal(1) / Decimal(7)
>>> from decimal import *
>>> getcontext()
Context(prec=28, rounding=ROUND_HALF_EVEN, Emin=-999999999, Emax=999999999,
- capitals=1, flags=[], traps=[Overflow, InvalidOperation,
- DivisionByZero])
+ capitals=1, flags=[], traps=[Overflow, DivisionByZero,
+ InvalidOperation])
>>> getcontext().prec = 7 # Set a new precision
serves as an explicit reminder of the details of the conversion (including
representation error). Decimal numbers include special values such as
:const:`NaN` which stands for "Not a number", positive and negative
-:const:`Infinity`, and :const:`-0`. ::
+:const:`Infinity`, and :const:`-0`.
>>> Decimal(10)
Decimal('10')
The significance of a new Decimal is determined solely by the number of digits
input. Context precision and rounding only come into play during arithmetic
-operations. ::
+operations.
+
+.. doctest:: newcontext
>>> getcontext().prec = 6
>>> Decimal('3.0')
Decimal('5.85988')
Decimals interact well with much of the rest of Python. Here is a small decimal
-floating point flying circus::
+floating point flying circus:
+
+.. doctest::
+ :options: +NORMALIZE_WHITESPACE
>>> data = map(Decimal, '1.34 1.87 3.45 2.35 1.00 0.03 9.25'.split())
>>> max(data)
>>> c % a
Decimal('0.77')
-And some mathematical functions are also available to Decimal::
+And some mathematical functions are also available to Decimal:
>>> Decimal(2).sqrt()
Decimal('1.414213562373095048801688724')
The :meth:`quantize` method rounds a number to a fixed exponent. This method is
useful for monetary applications that often round results to a fixed number of
-places::
+places:
>>> Decimal('7.325').quantize(Decimal('.01'), rounding=ROUND_DOWN)
Decimal('7.32')
In accordance with the standard, the :mod:`Decimal` module provides two ready to
use standard contexts, :const:`BasicContext` and :const:`ExtendedContext`. The
former is especially useful for debugging because many of the traps are
-enabled::
+enabled:
+
+.. doctest:: newcontext
+ :options: +NORMALIZE_WHITESPACE
>>> myothercontext = Context(prec=60, rounding=ROUND_HALF_DOWN)
>>> setcontext(myothercontext)
Decimal('3.14159292')
>>> getcontext()
Context(prec=9, rounding=ROUND_HALF_EVEN, Emin=-999999999, Emax=999999999,
- capitals=1, flags=[Inexact, Rounded], traps=[])
+ capitals=1, flags=[Rounded, Inexact], traps=[])
The *flags* entry shows that the rational approximation to :const:`Pi` was
rounded (digits beyond the context precision were thrown away) and that the
result is inexact (some of the discarded digits were non-zero).
Individual traps are set using the dictionary in the :attr:`traps` field of a
-context::
+context:
+.. doctest:: newcontext
+
+ >>> setcontext(ExtendedContext)
>>> Decimal(1) / Decimal(0)
Decimal('Infinity')
>>> getcontext().traps[DivisionByZero] = 1
but the result gives a total ordering on :class:`Decimal`
instances. Two :class:`Decimal` instances with the same numeric
value but different representations compare unequal in this
- ordering::
+ ordering:
>>> Decimal('12.0').compare_total(Decimal('12'))
Decimal('-1')
.. method:: Decimal.copy_sign(other)
Return a copy of the first operand with the sign set to be the
- same as the sign of the second operand. For example::
+ same as the sign of the second operand. For example:
>>> Decimal('2.3').copy_sign(Decimal('-1.5'))
Decimal('-2.3')
needed by the application. Another benefit is that rounding immediately
eliminates unintended effects from digits beyond the current precision. In the
following example, using unrounded inputs means that adding zero to a sum can
- change the result::
+ change the result:
+
+ .. doctest:: newcontext
>>> getcontext().prec = 3
>>> Decimal('3.4445') + Decimal('1.0023')
of nearly offsetting quantities resulting in loss of significance. Knuth
provides two instructive examples where rounded floating point arithmetic with
insufficient precision causes the breakdown of the associative and distributive
-properties of addition::
+properties of addition:
+
+.. doctest:: newcontext
# Examples from Seminumerical Algorithms, Section 4.2.2.
>>> from decimal import Decimal, getcontext
Decimal('0.0060000')
The :mod:`decimal` module makes it possible to restore the identities by
-expanding the precision sufficiently to avoid loss of significance::
+expanding the precision sufficiently to avoid loss of significance:
+
+.. doctest:: newcontext
>>> getcontext().prec = 20
>>> u, v, w = Decimal(11111113), Decimal(-11111111), Decimal('7.51111111')
various representations of zero with differing precisions yet equivalent in
value. This takes a bit of getting used to. For an eye accustomed to
normalized floating point representations, it is not immediately obvious that
-the following calculation returns a value equal to zero::
+the following calculation returns a value equal to zero:
>>> 1 / Decimal('Infinity')
Decimal('0E-1000000026')
Q. It is cumbersome to type ``decimal.Decimal('1234.5')``. Is there a way to
minimize typing when using the interactive interpreter?
-A. Some users abbreviate the constructor to just a single letter::
+A. Some users abbreviate the constructor to just a single letter:
>>> D = decimal.Decimal
>>> D('1.23') + D('3.45')
and need to be validated. What methods should be used?
A. The :meth:`quantize` method rounds to a fixed number of decimal places. If
-the :const:`Inexact` trap is set, it is also useful for validation::
+the :const:`Inexact` trap is set, it is also useful for validation:
>>> TWOPLACES = Decimal(10) ** -2 # same as Decimal('0.01')
>>> Decimal('3.214').quantize(TWOPLACES, context=Context(traps=[Inexact]))
Traceback (most recent call last):
...
- Inexact: Changed in rounding
+ Inexact
Q. Once I have valid two place inputs, how do I maintain that invariant
throughout an application?
A. Some operations like addition, subtraction, and multiplication by an integer
will automatically preserve fixed point. Others operations, like division and
non-integer multiplication, will change the number of decimal places and need to
-be followed-up with a :meth:`quantize` step::
+be followed-up with a :meth:`quantize` step:
>>> a = Decimal('102.72') # Initial fixed-point values
>>> b = Decimal('3.17')
Decimal('0.03')
In developing fixed-point applications, it is convenient to define functions
-to handle the :meth:`quantize` step::
+to handle the :meth:`quantize` step:
>>> def mul(x, y, fp=TWOPLACES):
... return (x * y).quantize(fp)
canonical value?
A. The :meth:`normalize` method maps all equivalent values to a single
-representative::
+representative:
>>> values = map(Decimal, '200 200.000 2E2 .02E+4'.split())
>>> [v.normalize() for v in values]
If an application does not care about tracking significance, it is easy to
remove the exponent and trailing zeroes, losing significance, but keeping the
-value unchanged::
+value unchanged:
>>> def remove_exponent(d):
... return d.quantize(Decimal(1)) if d == d.to_integral() else d.normalize()
A. Yes, all binary floating point numbers can be exactly expressed as a
Decimal. An exact conversion may take more precision than intuition would
-suggest, so we trap :const:`Inexact` to signal a need for more precision::
+suggest, so we trap :const:`Inexact` to signal a need for more precision:
+
+.. testcode::
def float_to_decimal(f):
"Convert a floating point number to a Decimal with no loss of information"
except Inexact:
ctx.prec += 1
+.. doctest::
+
>>> float_to_decimal(math.pi)
Decimal('3.141592653589793115997963468544185161590576171875')
A. There is some question about whether it is advisable to mix binary and
decimal floating point. Also, its use requires some care to avoid the
-representation issues associated with binary floating point::
+representation issues associated with binary floating point:
>>> float_to_decimal(1.1)
Decimal('1.100000000000000088817841970012523233890533447265625')
A. Yes. The principle is that all values are considered to be exact and so is
the arithmetic on those values. Only the results are rounded. The advantage
for inputs is that "what you type is what you get". A disadvantage is that the
-results can look odd if you forget that the inputs haven't been rounded::
+results can look odd if you forget that the inputs haven't been rounded:
+
+.. doctest:: newcontext
>>> getcontext().prec = 3
- >>> Decimal('3.104') + D('2.104')
+ >>> Decimal('3.104') + Decimal('2.104')
Decimal('5.21')
- >>> Decimal('3.104') + D('0.000') + D('2.104')
+ >>> Decimal('3.104') + Decimal('0.000') + Decimal('2.104')
Decimal('5.20')
The solution is either to increase precision or to force rounding of inputs
-using the unary plus operation::
+using the unary plus operation:
+
+.. doctest:: newcontext
>>> getcontext().prec = 3
>>> +Decimal('1.23456789') # unary plus triggers rounding
Decimal('1.23')
Alternatively, inputs can be rounded upon creation using the
-:meth:`Context.create_decimal` method::
+:meth:`Context.create_decimal` method:
>>> Context(prec=5, rounding=ROUND_DOWN).create_decimal('1.2345678')
Decimal('1.2345')
-
:mod:`difflib` --- Helpers for computing deltas
===============================================
.. sectionauthor:: Tim Peters <tim_one@users.sourceforge.net>
.. Markup by Fred L. Drake, Jr. <fdrake@acm.org>
+.. testsetup::
+ import sys
+ from difflib import *
This module provides classes and functions for comparing sequences. It
can be used for example, for comparing files, and can produce difference
expressed in the format returned by :func:`time.ctime`. If not specified, the
strings default to blanks.
- ::
-
>>> s1 = ['bacon\n', 'eggs\n', 'ham\n', 'guido\n']
>>> s2 = ['python\n', 'eggy\n', 'hamster\n', 'guido\n']
>>> for line in context_diff(s1, s2, fromfile='before.py', tofile='after.py'):
- ... sys.stdout.write(line)
+ ... sys.stdout.write(line) # doctest: +NORMALIZE_WHITESPACE
*** before.py
--- after.py
***************
Possibilities that don't score at least that similar to *word* are ignored.
The best (no more than *n*) matches among the possibilities are returned in a
- list, sorted by similarity score, most similar first. ::
+ list, sorted by similarity score, most similar first.
>>> get_close_matches('appel', ['ape', 'apple', 'peach', 'puppy'])
['apple', 'ape']
function :func:`IS_CHARACTER_JUNK`, which filters out whitespace characters (a
blank or tab; note: bad idea to include newline in this!).
- :file:`Tools/scripts/ndiff.py` is a command-line front-end to this function. ::
+ :file:`Tools/scripts/ndiff.py` is a command-line front-end to this function.
>>> diff = ndiff('one\ntwo\nthree\n'.splitlines(1),
... 'ore\ntree\nemu\n'.splitlines(1))
lines originating from file 1 or 2 (parameter *which*), stripping off line
prefixes.
- Example::
+ Example:
>>> diff = ndiff('one\ntwo\nthree\n'.splitlines(1),
... 'ore\ntree\nemu\n'.splitlines(1))
expressed in the format returned by :func:`time.ctime`. If not specified, the
strings default to blanks.
- ::
>>> s1 = ['bacon\n', 'eggs\n', 'ham\n', 'guido\n']
>>> s2 = ['python\n', 'eggy\n', 'hamster\n', 'guido\n']
>>> for line in unified_diff(s1, s2, fromfile='before.py', tofile='after.py'):
- ... sys.stdout.write(line)
+ ... sys.stdout.write(line) # doctest: +NORMALIZE_WHITESPACE
--- before.py
+++ after.py
@@ -1,4 +1,4 @@
conditions, the additional conditions ``k >= k'``, ``i <= i'``, and if ``i ==
i'``, ``j <= j'`` are also met. In other words, of all maximal matching blocks,
return one that starts earliest in *a*, and of all those maximal matching blocks
- that start earliest in *a*, return the one that starts earliest in *b*. ::
+ that start earliest in *a*, return the one that starts earliest in *b*.
>>> s = SequenceMatcher(None, " abcd", "abcd abcd")
>>> s.find_longest_match(0, 5, 0, 9)
- (0, 4, 5)
+ Match(a=0, b=4, size=5)
If *isjunk* was provided, first the longest matching block is determined as
above, but with the additional restriction that no junk element appears in the
Here's the same example as before, but considering blanks to be junk. That
prevents ``' abcd'`` from matching the ``' abcd'`` at the tail end of the second
sequence directly. Instead only the ``'abcd'`` can match, and matches the
- leftmost ``'abcd'`` in the second sequence::
+ leftmost ``'abcd'`` in the second sequence:
>>> s = SequenceMatcher(lambda x: x==" ", " abcd", "abcd abcd")
>>> s.find_longest_match(0, 5, 0, 9)
- (1, 0, 4)
+ Match(a=1, b=0, size=4)
If no blocks match, this returns ``(alo, blo, 0)``.
.. XXX Explain why a dummy is used!
- ::
+ .. doctest::
>>> s = SequenceMatcher(None, "abxcd", "abcd")
>>> s.get_matching_blocks()
- [(0, 0, 2), (3, 2, 2), (5, 4, 0)]
+ [Match(a=0, b=0, size=2), Match(a=3, b=2, size=2), Match(a=5, b=4, size=0)]
.. method:: SequenceMatcher.get_opcodes()
| | are equal). |
+---------------+---------------------------------------------+
- For example::
+ For example:
>>> a = "qabxcd"
>>> b = "abycdf"
The three methods that return the ratio of matching to total characters can give
different results due to differing levels of approximation, although
:meth:`quick_ratio` and :meth:`real_quick_ratio` are always at least as large as
-:meth:`ratio`::
+:meth:`ratio`:
>>> s = SequenceMatcher(None, "abcd", "bcde")
>>> s.ratio()
SequenceMatcher Examples
------------------------
-This example compares two strings, considering blanks to be "junk:" ::
+This example compares two strings, considering blanks to be "junk:"
>>> s = SequenceMatcher(lambda x: x == " ",
... "private Thread currentThread;",
:meth:`ratio` returns a float in [0, 1], measuring the similarity of the
sequences. As a rule of thumb, a :meth:`ratio` value over 0.6 means the
-sequences are close matches::
+sequences are close matches:
>>> print(round(s.ratio(), 3))
0.866
If you're only interested in where the sequences match,
-:meth:`get_matching_blocks` is handy::
+:meth:`get_matching_blocks` is handy:
>>> for block in s.get_matching_blocks():
... print("a[%d] and b[%d] match for %d elements" % block)
a[0] and b[0] match for 8 elements
- a[8] and b[17] match for 6 elements
- a[14] and b[23] match for 15 elements
+ a[8] and b[17] match for 21 elements
a[29] and b[38] match for 0 elements
Note that the last tuple returned by :meth:`get_matching_blocks` is always a
tuple element (number of elements matched) is ``0``.
If you want to know how to change the first sequence into the second, use
-:meth:`get_opcodes`::
+:meth:`get_opcodes`:
>>> for opcode in s.get_opcodes():
... print("%6s a[%d:%d] b[%d:%d]" % opcode)
equal a[0:8] b[0:8]
insert a[8:8] b[8:17]
- equal a[8:14] b[17:23]
- equal a[14:29] b[23:38]
+ equal a[8:29] b[17:38]
See also the function :func:`get_close_matches` in this module, which shows how
simple code building on :class:`SequenceMatcher` can be used to do useful work.
This example compares two texts. First we set up the texts, sequences of
individual single-line strings ending with newlines (such sequences can also be
-obtained from the :meth:`readlines` method of file-like objects)::
+obtained from the :meth:`readlines` method of file-like objects):
>>> text1 = ''' 1. Beautiful is better than ugly.
... 2. Explicit is better than implicit.
... 5. Flat is better than nested.
... '''.splitlines(1)
-Next we instantiate a Differ object::
+Next we instantiate a Differ object:
>>> d = Differ()
filter out line and character "junk." See the :meth:`Differ` constructor for
details.
-Finally, we compare the two::
+Finally, we compare the two:
>>> result = list(d.compare(text1, text2))
-``result`` is a list of strings, so let's pretty-print it::
+``result`` is a list of strings, so let's pretty-print it:
>>> from pprint import pprint
>>> pprint(result)
'- 2. Explicit is better than implicit.\n',
'- 3. Simple is better than complex.\n',
'+ 3. Simple is better than complex.\n',
- '? ++ \n',
+ '? ++\n',
'- 4. Complex is better than complicated.\n',
- '? ^ ---- ^ \n',
+ '? ^ ---- ^\n',
'+ 4. Complicated is better than complex.\n',
- '? ++++ ^ ^ \n',
+ '? ++++ ^ ^\n',
'+ 5. Flat is better than nested.\n']
-As a single multi-line string it looks like this::
+As a single multi-line string it looks like this:
>>> import sys
>>> sys.stdout.writelines(result)
It is also contained in the Python source distribution, as
:file:`Tools/scripts/diff.py`.
-::
+.. testcode::
""" Command line interface to difflib.py providing diffs in four formats:
Return the shell-style *pattern* converted to a regular expression.
- Example::
+ Example:
>>> import fnmatch, re
>>>
Finds and returns the closest :class:`Fraction` to ``self`` that
has denominator at most max_denominator. This method is useful for
- finding rational approximations to a given floating-point number::
+ finding rational approximations to a given floating-point number:
+ >>> from fractions import Fraction
>>> Fraction('3.1415926535897932').limit_denominator(1000)
- Fraction(355, 113)
+ Fraction(355L, 113L)
- or for recovering a rational number that's represented as a float::
+ or for recovering a rational number that's represented as a float:
>>> from math import pi, cos
>>> Fraction.from_float(cos(pi/3))
Fraction(4503599627370497L, 9007199254740992L)
>>> Fraction.from_float(cos(pi/3)).limit_denominator()
- Fraction(1, 2)
+ Fraction(1L, 2L)
.. method:: Fraction.__floor__()
Module :mod:`numbers`
The abstract base classes making up the numeric tower.
-
class's attributes, and recursively of the attributes of its class's base
classes.
- The resulting list is sorted alphabetically.
+ The resulting list is sorted alphabetically. For example:
+
+ >>> import struct
+ >>> dir() # doctest: +SKIP
+ ['__builtins__', '__doc__', '__name__', 'struct']
+ >>> dir(struct) # doctest: +NORMALIZE_WHITESPACE
+ ['Struct', '__builtins__', '__doc__', '__file__', '__name__',
+ '__package__', '_clearcache', 'calcsize', 'error', 'pack', 'pack_into',
+ 'unpack', 'unpack_from']
+ >>> class Foo(object):
+ ... def __dir__(self):
+ ... return ["kan", "ga", "roo"]
+ ...
+ >>> f = Foo()
+ >>> dir(f)
+ ['ga', 'kan', 'roo']
.. note::
iterator returned by :func:`enumerate` returns a tuple containing a count (from
zero) and the corresponding value obtained from iterating over *iterable*.
:func:`enumerate` is useful for obtaining an indexed series: ``(0, seq[0])``,
- ``(1, seq[1])``, ``(2, seq[2])``, .... For example::
+ ``(1, seq[1])``, ``(2, seq[2])``, .... For example:
>>> for i, season in enumerate(['Spring', 'Summer', 'Fall', 'Winter')]:
- >>> print(i, season)
+ ... print(i, season)
0 Spring
1 Summer
2 Fall
propagated. If the *locals* dictionary is omitted it defaults to the *globals*
dictionary. If both dictionaries are omitted, the expression is executed in the
environment where :func:`eval` is called. The return value is the result of
- the evaluated expression. Syntax errors are reported as exceptions. Example::
+ the evaluated expression. Syntax errors are reported as exceptions. Example:
>>> x = 1
>>> eval('x+1')
.. XXX does accept objects with __index__ too
.. function:: range([start,] stop[, step])
- This is a versatile function to create iterators containing arithmetic
- progressions. It is most often used in :keyword:`for` loops. The arguments
- must be integers. If the *step* argument is omitted, it defaults to ``1``.
- If the *start* argument is omitted, it defaults to ``0``. The full form
- returns an iterator of plain integers ``[start, start + step, start + 2 *
- step, ...]``. If *step* is positive, the last element is the largest ``start
- + i * step`` less than *stop*; if *step* is negative, the last element is the
- smallest ``start + i * step`` greater than *stop*. *step* must not be zero
- (or else :exc:`ValueError` is raised). Example::
+ This is a versatile function to create lists containing arithmetic progressions.
+ It is most often used in :keyword:`for` loops. The arguments must be plain
+ integers. If the *step* argument is omitted, it defaults to ``1``. If the
+ *start* argument is omitted, it defaults to ``0``. The full form returns a list
+ of plain integers ``[start, start + step, start + 2 * step, ...]``. If *step*
+ is positive, the last element is the largest ``start + i * step`` less than
+ *stop*; if *step* is negative, the last element is the smallest ``start + i *
+ step`` greater than *stop*. *step* must not be zero (or else :exc:`ValueError`
+ is raised). Example:
>>> list(range(10))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
:noindex:
Return a new type object. This is essentially a dynamic form of the
- :keyword:`class` statement. The *name* string is the class name and becomes
- the :attr:`__name__` attribute; the *bases* tuple itemizes the base classes
- and becomes the :attr:`__bases__` attribute; and the *dict* dictionary is the
- namespace containing definitions for class body and becomes the
- :attr:`__dict__` attribute. For example, the following two statements create
- identical :class:`type` objects::
+ :keyword:`class` statement. The *name* string is the class name and becomes the
+ :attr:`__name__` attribute; the *bases* tuple itemizes the base classes and
+ becomes the :attr:`__bases__` attribute; and the *dict* dictionary is the
+ namespace containing definitions for class body and becomes the :attr:`__dict__`
+ attribute. For example, the following two statements create identical
+ :class:`type` objects:
>>> class X(object):
... a = 1
some portion of a function's arguments and/or keywords resulting in a new object
with a simplified signature. For example, :func:`partial` can be used to create
a callable that behaves like the :func:`int` function where the *base* argument
- defaults to two::
+ defaults to two:
+ >>> from functools import partial
>>> basetwo = partial(int, base=2)
>>> basetwo.__doc__ = 'Convert base 2 string to an int.'
>>> basetwo('10010')
This is a convenience function for invoking ``partial(update_wrapper,
wrapped=wrapped, assigned=assigned, updated=updated)`` as a function decorator
- when defining a wrapper function. For example::
+ when defining a wrapper function. For example:
+ >>> from functools import wraps
>>> def my_decorator(f):
... @wraps(f)
... def wrapper(*args, **kwds):
Alias for :exc:`GetoptError`; for backward compatibility.
-An example using only Unix style options::
+An example using only Unix style options:
>>> import getopt
>>> args = '-a -b -cfoo -d bar a1 a2'.split()
>>> args
['a1', 'a2']
-Using long option names is equally easy::
+Using long option names is equally easy:
>>> s = '--condition=foo --testing --output-file abc.def -x a1 a2'
>>> args = s.split()
>>> optlist, args = getopt.getopt(args, 'x', [
... 'condition=', 'output-file=', 'testing'])
>>> optlist
- [('--condition', 'foo'), ('--testing', ''), ('--output-file', 'abc.def'), ('-x',
- '')]
+ [('--condition', 'foo'), ('--testing', ''), ('--output-file', 'abc.def'), ('-x', '')]
>>> args
['a1', 'a2']
>>> m.block_size
64
-More condensed::
+More condensed:
>>> hashlib.sha224(b"Nobody inspects the spammish repetition").hexdigest()
b'a4337bc45a8fc544c03f52dc550cd6e1e87021bc896588bd79e901e2'
hashes as well as any other algorithms that your OpenSSL library may offer. The
named constructors are much faster than :func:`new` and should be preferred.
-Using :func:`new` with an algorithm provided by OpenSSL::
+Using :func:`new` with an algorithm provided by OpenSSL:
>>> h = hashlib.new('ripemd160')
>>> h.update(b"Nobody inspects the spammish repetition")
if item > heap[0]:
item = heapreplace(heap, item)
-Example of use::
+Example of use:
>>> from heapq import heappush, heappop
>>> heap = []
>>> data.sort()
>>> data == ordered
True
- >>>
The module also offers three general purpose functions based on heaps.
.. sectionauthor:: Raymond Hettinger <python@rcn.com>
+.. testsetup::
+
+ from itertools import *
+
+
This module implements a number of :term:`iterator` building blocks inspired by
constructs from the Haskell and SML programming languages. Each has been recast
in a form suitable for Python.
n = len(pool)
r = n if r is None else r
indices = range(n)
- cycles = range(n-r+1, n+1)[::-1]
+ cycles = range(n, n-r, -1)
yield tuple(pool[i] for i in indices[:r])
while n:
for i in reversed(range(r)):
--------
The following examples show common uses for each tool and demonstrate ways they
-can be combined. ::
+can be combined.
+
+.. doctest::
# Show a dictionary sorted and grouped by value
>>> from operator import itemgetter
kept small by linking the tools together in a functional style which helps
eliminate temporary variables. High speed is retained by preferring
"vectorized" building blocks over the use of for-loops and :term:`generator`\s
-which incur interpreter overhead. ::
+which incur interpreter overhead.
+
+.. testcode::
def take(n, seq):
return list(islice(seq, n))
.. sectionauthor:: Skip Montanaro <skip@automatrix.com>
+.. testsetup::
+
+ import operator
+ from operator import itemgetter
+
The :mod:`operator` module exports a set of functions implemented in C
corresponding to the intrinsic operators of Python. For example,
Be careful not to misinterpret the results of these functions; none have any
measure of reliability with instance objects.
- For example::
+ For example:
>>> class C:
... pass
useful than it otherwise might be.
Example: Build a dictionary that maps the ordinals from ``0`` to ``255`` to
-their character equivalents. ::
+their character equivalents.
- >>> import operator
>>> d = {}
>>> keys = range(256)
>>> vals = map(chr, keys)
- >>> map(operator.setitem, [d]*len(keys), keys, vals)
+ >>> map(operator.setitem, [d]*len(keys), keys, vals) # doctest: +SKIP
.. XXX: find a better, readable, example
The items can be any type accepted by the operand's :meth:`__getitem__`
method. Dictionaries accept any hashable value. Lists, tuples, and
- strings accept an index or a slice::
+ strings accept an index or a slice:
- >>> itemgetter(1)('ABCDEFG')
- 'B'
- >>> itemgetter(1,3,5)('ABCDEFG')
- ('B', 'D', 'F')
- >>> itemgetter(slice(2,None))('ABCDEFG')
- 'CDEFG'
+ >>> itemgetter(1)('ABCDEFG')
+ 'B'
+ >>> itemgetter(1,3,5)('ABCDEFG')
+ ('B', 'D', 'F')
+ >>> itemgetter(slice(2,None))('ABCDEFG')
+ 'CDEFG'
.. versionadded:: 2.4
Example of using :func:`itemgetter` to retrieve specific fields from a
- tuple record::
+ tuple record:
- >>> from operator import itemgetter
>>> inventory = [('apple', 3), ('banana', 2), ('pear', 5), ('orange', 1)]
>>> getcount = itemgetter(1)
>>> map(getcount, inventory)
the depth of the objects being formatted. The desired output width is
constrained using the *width* parameter; the default is 80 characters. If a
structure cannot be formatted within the constrained width, a best effort will
- be made. ::
+ be made.
>>> import pprint
>>> stuff = ['spam', 'eggs', 'lumberjack', 'knights', 'ni']
>>> stuff.insert(0, stuff[:])
>>> pp = pprint.PrettyPrinter(indent=4)
>>> pp.pprint(stuff)
- [ ['spam', 'eggs', 'lumberjack', 'knights', 'ni'],
+ [ [ 'spam', 'eggs', 'lumberjack', 'knights', 'ni'],
'spam',
'eggs',
'lumberjack',
in the interactive interpreter instead of the :func:`print` function for
inspecting values (you can even reassign ``print = pprint.pprint`` for use
within a scope). *indent*, *width* and *depth* will be passed to the
- :class:`PrettyPrinter` constructor as formatting parameters. ::
+ :class:`PrettyPrinter` constructor as formatting parameters.
>>> import pprint
>>> stuff = ['spam', 'eggs', 'lumberjack', 'knights', 'ni']
>>> stuff.insert(0, stuff)
>>> pprint.pprint(stuff)
- [<Recursion on list with id=869440>,
- '',
- '/usr/local/lib/python1.5',
- '/usr/local/lib/python1.5/test',
- '/usr/local/lib/python1.5/sunos5',
- '/usr/local/lib/python1.5/sharedmodules',
- '/usr/local/lib/python1.5/tkinter']
+ [<Recursion on list with id=...>,
+ 'spam',
+ 'eggs',
+ 'lumberjack',
+ 'knights',
+ 'ni']
.. function:: isreadable(object)
Determine if the formatted representation of *object* is "readable," or can be
used to reconstruct the value using :func:`eval`. This always returns ``False``
- for recursive objects. ::
+ for recursive objects.
>>> pprint.isreadable(stuff)
False
Determine if *object* requires a recursive representation.
-One more support function is also defined:
+One more support function is also defined:
.. function:: saferepr(object)
recursive reference will be represented as ``<Recursion on typename with
id=number>``. The representation is not otherwise formatted.
-::
-
>>> pprint.saferepr(stuff)
- "[<Recursion on list with id=682968>, '', '/usr/local/lib/python1.5', '/usr/loca
- l/lib/python1.5/test', '/usr/local/lib/python1.5/sunos5', '/usr/local/lib/python
- 1.5/sharedmodules', '/usr/local/lib/python1.5/tkinter']"
+ "[<Recursion on list with id=...>, 'spam', 'eggs', 'lumberjack', 'knights', 'ni']"
.. _prettyprinter-objects:
Return a random floating point number *N* such that ``a <= N < b``.
+.. function:: triangular(low, high, mode)
+
+ Return a random floating point number *N* such that ``low <= N < high``
+ and with the specified *mode* between those bounds.
+
+ If *mode* is not specified or is ``None``, it defaults to the midpoint
+ between the upper and lower bounds, producing a symmetric distribution.
+
+ The default values for *low* and *high* are zero and one.
.. function:: betavariate(alpha, beta)
``abc`` or ``a|b`` are allowed, but ``a*`` and ``a{3,4}`` are not. Note that
patterns which start with positive lookbehind assertions will never match at the
beginning of the string being searched; you will most likely want to use the
- :func:`search` function rather than the :func:`match` function::
+ :func:`search` function rather than the :func:`match` function:
>>> import re
>>> m = re.search('(?<=abc)def', 'abcdef')
>>> m.group(0)
'def'
- This example looks for a word following a hyphen::
+ This example looks for a word following a hyphen:
>>> m = re.search('(?<=-)\w+', 'spam-egg')
>>> m.group(0)
:const:`MULTILINE` mode also immediately following a newline. The "match"
operation succeeds only if the pattern matches at the start of the string
regardless of mode, or at the starting position given by the optional *pos*
-argument regardless of whether a newline precedes it. ::
+argument regardless of whether a newline precedes it.
- >>> re.match("c", "abcdef") # No match
- >>> re.search("c", "abcdef")
- <_sre.SRE_Match object at 0x827e9c0> # Match
+ >>> re.match("c", "abcdef") # No match
+ >>> re.search("c", "abcdef") # Match
+ <_sre.SRE_Match object at ...>
.. _contents-of-module-re:
If there are capturing groups in the separator and it matches at the start of
the string, the result will start with an empty string. The same holds for
- the end of the string::
+ the end of the string:
>>> re.split('(\W+)', '...words, words...')
['', '...', 'words', ', ', 'words', '...', '']
in the separator, the 0th, the 2nd and so forth).
Note that *split* will never split a string on an empty pattern match.
- For example::
+ For example:
>>> re.split('x*', 'foo')
['foo']
converted to a single newline character, ``\r`` is converted to a linefeed, and
so forth. Unknown escapes such as ``\j`` are left alone. Backreferences, such
as ``\6``, are replaced with the substring matched by group 6 in the pattern.
- For example::
+ For example:
>>> re.sub(r'def\s+([a-zA-Z_][a-zA-Z_0-9]*)\s*\(\s*\):',
... r'static PyObject*\npy_\1(void)\n{',
If *repl* is a function, it is called for every non-overlapping occurrence of
*pattern*. The function takes a single match object argument, and returns the
- replacement string. For example::
+ replacement string. For example:
>>> def dashrepl(matchobj):
... if matchobj.group(0) == '-': return ' '
from *pos* to ``endpos - 1`` will be searched for a match. If *endpos* is less
than *pos*, no match will be found, otherwise, if *rx* is a compiled regular
expression object, ``rx.match(string, 0, 50)`` is equivalent to
- ``rx.match(string[:50], 0)``. ::
+ ``rx.match(string[:50], 0)``.
>>> pattern = re.compile("o")
>>> pattern.match("dog") # No match as "o" is not at the start of "dog."
>>> pattern.match("dog", 1) # Match as "o" is the 2nd character of "dog".
- <_sre.SRE_Match object at 0x827eb10>
+ <_sre.SRE_Match object at ...>
.. method:: RegexObject.search(string[, pos[, endpos]])
pattern, an :exc:`IndexError` exception is raised. If a group is contained in a
part of the pattern that did not match, the corresponding result is ``None``.
If a group is contained in a part of the pattern that matched multiple times,
- the last match is returned. ::
+ the last match is returned.
>>> m = re.match(r"(\w+) (\w+)", "Isaac Newton, physicist")
- >>> m.group(0)
- 'Isaac Newton' # The entire match
- >>> m.group(1)
- 'Isaac' # The first parenthesized subgroup.
- >>> m.group(2)
- 'Newton' # The second parenthesized subgroup.
- >>> m.group(1, 2)
- ('Isaac', 'Newton') # Multiple arguments give us a tuple.
+ >>> m.group(0) # The entire match
+ 'Isaac Newton'
+ >>> m.group(1) # The first parenthesized subgroup.
+ 'Isaac'
+ >>> m.group(2) # The second parenthesized subgroup.
+ 'Newton'
+ >>> m.group(1, 2) # Multiple arguments give us a tuple.
+ ('Isaac', 'Newton')
If the regular expression uses the ``(?P<name>...)`` syntax, the *groupN*
arguments may also be strings identifying groups by their group name. If a
string argument is not used as a group name in the pattern, an :exc:`IndexError`
exception is raised.
- A moderately complicated example::
+ A moderately complicated example:
>>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcom Reynolds")
>>> m.group('first_name')
>>> m.group('last_name')
'Reynolds'
- Named groups can also be referred to by their index::
+ Named groups can also be referred to by their index:
>>> m.group(1)
'Malcom'
>>> m.group(2)
'Reynolds'
- If a group matches multiple times, only the last match is accessible::
+ If a group matches multiple times, only the last match is accessible:
+
>>> m = re.match(r"(..)+", "a1b2c3") # Matches 3 times.
>>> m.group(1) # Returns only the last match.
'c3'
many groups are in the pattern. The *default* argument is used for groups that
did not participate in the match; it defaults to ``None``.
- For example::
+ For example:
>>> m = re.match(r"(\d+)\.(\d+)", "24.1632")
>>> m.groups()
If we make the decimal place and everything after it optional, not all groups
might participate in the match. These groups will default to ``None`` unless
- the *default* argument is given::
+ the *default* argument is given:
>>> m = re.match(r"(\d+)\.?(\d+)?", "24")
- >>> m.groups()
- ('24', None) # Second group defaults to None.
- >>> m.groups('0')
- ('24', '0') # Now, the second group defaults to '0'.
+ >>> m.groups() # Second group defaults to None.
+ ('24', None)
+ >>> m.groups('0') # Now, the second group defaults to '0'.
+ ('24', '0')
.. method:: MatchObject.groupdict([default])
Return a dictionary containing all the *named* subgroups of the match, keyed by
the subgroup name. The *default* argument is used for groups that did not
- participate in the match; it defaults to ``None``. For example::
+ participate in the match; it defaults to ``None``. For example:
>>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcom Reynolds")
>>> m.groupdict()
``m.start(0)`` is 1, ``m.end(0)`` is 2, ``m.start(1)`` and ``m.end(1)`` are both
2, and ``m.start(2)`` raises an :exc:`IndexError` exception.
- An example that will remove *remove_this* from email addresses::
+ An example that will remove *remove_this* from email addresses:
>>> email = "tony@tiremove_thisger.net"
>>> m = re.search("remove_this", email)
^^^^^^^^^^^^^^^^^^^
In this example, we'll use the following helper function to display match
-objects a little more gracefully::
+objects a little more gracefully:
+
+.. testcode::
def displaymatch(match):
if match is None:
for king, "q" for queen, j for jack, "0" for 10, and "1" through "9"
representing the card with that value.
-To see if a given string is a valid hand, one could do the following::
+To see if a given string is a valid hand, one could do the following:
- >>> valid = re.compile(r"[0-9akqj]{5}$"
+ >>> valid = re.compile(r"[0-9akqj]{5}$")
>>> displaymatch(valid.match("ak05q")) # Valid.
- <Match: 'ak05q', groups=()>
+ "<Match: 'ak05q', groups=()>"
>>> displaymatch(valid.match("ak05e")) # Invalid.
>>> displaymatch(valid.match("ak0")) # Invalid.
>>> displaymatch(valid.match("727ak")) # Valid.
- <Match: '727ak', groups=()>
+ "<Match: '727ak', groups=()>"
That last hand, ``"727ak"``, contained a pair, or two of the same valued cards.
-To match this with a regular expression, one could use backreferences as such::
+To match this with a regular expression, one could use backreferences as such:
>>> pair = re.compile(r".*(.).*\1")
>>> displaymatch(pair.match("717ak")) # Pair of 7s.
- <Match: '717', groups=('7',)>
+ "<Match: '717', groups=('7',)>"
>>> displaymatch(pair.match("718ak")) # No pairs.
>>> displaymatch(pair.match("354aa")) # Pair of aces.
- <Match: '345aa', groups=('a',)>
+ "<Match: '354aa', groups=('a',)>"
To find out what card the pair consists of, one could use the :func:`group`
-method of :class:`MatchObject` in the following manner::
+method of :class:`MatchObject` in the following manner:
+
+.. doctest::
>>> pair.match("717ak").group(1)
'7'
recursion, you may encounter a :exc:`RuntimeError` exception with the message
``maximum recursion limit`` exceeded. For example, ::
- >>> import re
>>> s = 'Begin ' + 1000*'a very long string ' + 'end'
>>> re.match('Begin (\w| )*? end', s).end()
Traceback (most recent call last):
In a nutshell, :func:`match` only attempts to match a pattern at the beginning
of a string where :func:`search` will match a pattern anywhere in a string.
-For example::
+For example:
>>> re.match("o", "dog") # No match as "o" is not the first letter of "dog".
>>> re.search("o", "dog") # Match as search() looks everywhere in the string.
- <_sre.SRE_Match object at 0x827e9f8>
+ <_sre.SRE_Match object at ...>
.. note::
- The following applies only to regular expression objects like those created
- with ``re.compile("pattern")``, not the primitives
- ``re.match(pattern, string)`` or ``re.search(pattern, string)``.
+ The following applies only to regular expression objects like those created
+ with ``re.compile("pattern")``, not the primitives ``re.match(pattern,
+ string)`` or ``re.search(pattern, string)``.
:func:`match` has an optional second parameter that gives an index in the string
-where the search is to start::
+where the search is to start:
>>> pattern = re.compile("o")
>>> pattern.match("dog") # No match as "o" is not at the start of "dog."
+
# Equivalent to the above expression as 0 is the default starting index:
>>> pattern.match("dog", 0)
+
# Match as "o" is the 2nd character of "dog" (index 0 is the first):
>>> pattern.match("dog", 1)
- <_sre.SRE_Match object at 0x827eb10>
+ <_sre.SRE_Match object at ...>
>>> pattern.match("dog", 2) # No match as "o" is not the 3rd character of "dog."
creates a phonebook.
First, here is the input. Normally it may come from a file, here we are using
-triple-quoted string syntax::
+triple-quoted string syntax:
>>> input = """Ross McFluff: 834.345.1254 155 Elm Street
-
- Ronald Heathmore: 892.345.3428 436 Finley Avenue
- Frank Burger: 925.541.7625 662 South Dogwood Way
-
-
- Heather Albrecht: 548.326.4584 919 Park Place"""
+ ...
+ ... Ronald Heathmore: 892.345.3428 436 Finley Avenue
+ ... Frank Burger: 925.541.7625 662 South Dogwood Way
+ ...
+ ...
+ ... Heather Albrecht: 548.326.4584 919 Park Place"""
The entries are separated by one or more newlines. Now we convert the string
-into a list with each nonempty line having its own entry::
+into a list with each nonempty line having its own entry:
+
+.. doctest::
+ :options: +NORMALIZE_WHITESPACE
>>> entries = re.split("\n+", input)
>>> entries
- ['Ross McFluff 834.345.1254 155 Elm Street',
- 'Ronald Heathmore 892.345.3428 436 Finley Avenue',
- 'Frank Burger 925.541.7625 662 South Dogwood Way',
- 'Heather Albrecht 548.326.4584 919 Park Place']
+ ['Ross McFluff: 834.345.1254 155 Elm Street',
+ 'Ronald Heathmore: 892.345.3428 436 Finley Avenue',
+ 'Frank Burger: 925.541.7625 662 South Dogwood Way',
+ 'Heather Albrecht: 548.326.4584 919 Park Place']
Finally, split each entry into a list with first name, last name, telephone
number, and address. We use the ``maxsplit`` parameter of :func:`split`
-because the address has spaces, our splitting pattern, in it::
+because the address has spaces, our splitting pattern, in it:
+
+.. doctest::
+ :options: +NORMALIZE_WHITESPACE
>>> [re.split(":? ", entry, 3) for entry in entries]
[['Ross', 'McFluff', '834.345.1254', '155 Elm Street'],
The ``:?`` pattern matches the colon after the last name, so that it does not
occur in the result list. With a ``maxsplit`` of ``4``, we could separate the
-house number from the street name::
+house number from the street name:
+
+.. doctest::
+ :options: +NORMALIZE_WHITESPACE
>>> [re.split(":? ", entry, 4) for entry in entries]
[['Ross', 'McFluff', '834.345.1254', '155', 'Elm Street'],
:func:`findall` matches *all* occurrences of a pattern, not just the first
one as :func:`search` does. For example, if one was a writer and wanted to
find all of the adverbs in some text, he or she might use :func:`findall` in
-the following manner::
+the following manner:
>>> text = "He was carefully disguised but captured quickly by police."
>>> re.findall(r"\w+ly", text)
text, :func:`finditer` is useful as it provides instances of
:class:`MatchObject` instead of strings. Continuing with the previous example,
if one was a writer who wanted to find all of the adverbs *and their positions*
-in some text, he or she would use :func:`finditer` in the following manner::
+in some text, he or she would use :func:`finditer` in the following manner:
>>> text = "He was carefully disguised but captured quickly by police."
>>> for m in re.finditer(r"\w+ly", text):
- print('%02d-%02d: %s' % (m.start(), m.end(), m.group(0)))
+ ... print('%02d-%02d: %s' % (m.start(), m.end(), m.group(0)))
07-16: carefully
40-47: quickly
Raw string notation (``r"text"``) keeps regular expressions sane. Without it,
every backslash (``'\'``) in a regular expression would have to be prefixed with
another one to escape it. For example, the two following lines of code are
-functionally identical::
+functionally identical:
>>> re.match(r"\W(.)\1\W", " ff ")
- <_sre.SRE_Match object at 0x8262760>
+ <_sre.SRE_Match object at ...>
>>> re.match("\\W(.)\\1\\W", " ff ")
- <_sre.SRE_Match object at 0x82627a0>
+ <_sre.SRE_Match object at ...>
When one wants to match a literal backslash, it must be escaped in the regular
expression. With raw string notation, this means ``r"\\"``. Without raw string
notation, one must use ``"\\\\"``, making the following lines of code
-functionally identical::
+functionally identical:
>>> re.match(r"\\", r"\\")
- <_sre.SRE_Match object at 0x827eb48>
+ <_sre.SRE_Match object at ...>
>>> re.match("\\\\", r"\\")
- <_sre.SRE_Match object at 0x827ec60>
+ <_sre.SRE_Match object at ...>
Example::
>>> import sched, time
- >>> s=sched.scheduler(time.time, time.sleep)
+ >>> s = sched.scheduler(time.time, time.sleep)
>>> def print_time(): print("From print_time", time.time())
...
>>> def print_some_times():
string, as would be printed by the C function :cfunc:`perror`.
-.. type:: epoll([sizehint=-1])
+.. function:: epoll([sizehint=-1])
- (Only supported on Linux 2.5.44 and newer.) Returns an edge polling
- object, which can be used as Edge or Level Triggered interface for I/O
- events; see section :ref:`epoll-objects` below for the methods supported
- by epolling objects.
+ (Only supported on Linux 2.5.44 and newer.) Returns an edge polling object,
+ which can be used as Edge or Level Triggered interface for I/O events; see
+ section :ref:`epoll-objects` below for the methods supported by epolling
+ objects.
.. versionadded:: 2.6
by polling objects.
-.. type:: kqueue()
+.. function:: kqueue()
- (Only supported on BSD.) Returns a kernel queue object
- object; see section :ref:`kqueue-objects` below for the methods supported
- by kqueue objects.
+ (Only supported on BSD.) Returns a kernel queue object object; see section
+ :ref:`kqueue-objects` below for the methods supported by kqueue objects.
.. versionadded:: 2.6
-.. type:: kqueue(ident, filter=KQ_FILTER_READ, flags=KQ_ADD, fflags=0, data=0, udata=0)
+.. function:: kqueue(ident, filter=KQ_FILTER_READ, flags=KQ_ADD, fflags=0, data=0, udata=0)
- (Only supported on BSD.) Returns a kernel event object
- object; see section :ref:`kevent-objects` below for the methods supported
- by kqueue objects.
+ (Only supported on BSD.) Returns a kernel event object object; see section
+ :ref:`kevent-objects` below for the methods supported by kqueue objects.
.. versionadded:: 2.6
Kevent Objects
--------------
- http://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2
+http://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2
- .. attribute:: ident
+.. attribute:: kevent.ident
Value used to identify the event. The interpretation depends on the filter
but it's usually the file descriptor. In the constructor ident can either
be an int or an object with a fileno() function. kevent stores the integer
internally.
- .. attribute:: filter
+.. attribute:: kevent.filter
Name of the kernel filter
| :const:`KQ_FILTER_TIMER` | Establishes an arbitrary timer |
+---------------------------+---------------------------------------------+
- .. attribute:: flags
+.. attribute:: kevent.flags
Filter action
+---------------------------+---------------------------------------------+
- .. attribute:: fflags
+.. attribute:: kevent.fflags
Filter specific flags
- *:const:`KQ_FILTER_READ` and :const:`KQ_FILTER_WRITE` filter flags
+ *:const:`KQ_FILTER_READ` and :const:`KQ_FILTER_WRITE` filter flags*
+----------------------------+--------------------------------------------+
| Constant | Meaning |
+----------------------------+--------------------------------------------+
- .. attribute:: data
+.. attribute:: kevent.data
Filter specific data
- .. attribute:: udata
+.. attribute:: kevent.udata
User defined value
Values of *n* less than ``0`` are treated as ``0`` (which yields an empty
sequence of the same type as *s*). Note also that the copies are shallow;
nested structures are not copied. This often haunts new Python programmers;
- consider::
+ consider:
>>> lists = [[]] * 3
>>> lists
[[3], [3], [3]]
What has happened is that ``[[]]`` is a one-element list containing an empty
- list, so all three elements of ``[[]] * 3`` are (pointers to) this single
- empty list. Modifying any of the elements of ``lists`` modifies this single
- list. You can create a list of different lists this way::
+ list, so all three elements of ``[[]] * 3`` are (pointers to) this single empty
+ list. Modifying any of the elements of ``lists`` modifies this single list.
+ You can create a list of different lists this way:
>>> lists = [[] for i in range(3)]
>>> lists[0].append(3)
Return a copy of the string with leading characters removed. The *chars*
argument is a string specifying the set of characters to be removed. If omitted
or ``None``, the *chars* argument defaults to removing whitespace. The *chars*
- argument is not a prefix; rather, all combinations of its values are stripped::
+ argument is not a prefix; rather, all combinations of its values are stripped:
>>> ' spacious '.lstrip()
'spacious '
Return a copy of the string with trailing characters removed. The *chars*
argument is a string specifying the set of characters to be removed. If omitted
or ``None``, the *chars* argument defaults to removing whitespace. The *chars*
- argument is not a suffix; rather, all combinations of its values are stripped::
+ argument is not a suffix; rather, all combinations of its values are stripped:
>>> ' spacious '.rstrip()
' spacious'
The *chars* argument is a string specifying the set of characters to be removed.
If omitted or ``None``, the *chars* argument defaults to removing whitespace.
The *chars* argument is not a prefix or suffix; rather, all combinations of its
- values are stripped::
+ values are stripped:
>>> ' spacious '.strip()
'spacious'
A *map* for :meth:`translate` is usually best created by
:meth:`str.maketrans`.
+ You can use the :func:`maketrans` helper function in the :mod:`string` module to
+ create a translation table. For string objects, set the *table* argument to
+ ``None`` for translations that only delete characters:
+
.. note::
An even more flexible approach is to create a custom character mapping
When the right argument is a dictionary (or other mapping type), then the
formats in the string *must* include a parenthesised mapping key into that
dictionary inserted immediately after the ``'%'`` character. The mapping key
-selects the value to be formatted from the mapping. For example::
+selects the value to be formatted from the mapping. For example:
+
- >>> print('%(language)s has %(#)03d quote types.' %
- {'language': "Python", "#": 2})
+ >>> print('%(language)s has %(#)03d quote types.' % \
+ ... {'language': "Python", "#": 2})
Python has 002 quote types.
In this case no ``*`` specifiers may occur in a format (since they require a
This is the object passed to the constructor's *template* argument. In general,
you shouldn't change it, but read-only access is not enforced.
-Here is an example of how to use a Template::
+Here is an example of how to use a Template:
>>> from string import Template
>>> s = Template('$who likes $what')
The default values used to fill in any missing data when more accurate values
cannot be inferred are ``(1900, 1, 1, 0, 0, 0, 0, 1, -1)``.
- For example::
+ For example:
>>> import time
- >>> time.strptime("30 Nov 00", "%d %b %y")
- (2000, 11, 30, 0, 0, 0, 3, 335, -1)
+ >>> time.strptime("30 Nov 00", "%d %b %y") # doctest: +NORMALIZE_WHITESPACE
+ time.struct_time(tm_year=2000, tm_mon=11, tm_mday=30, tm_hour=0, tm_min=0,
+ tm_sec=0, tm_wday=3, tm_yday=335, tm_isdst=-1)
Support for the ``%Z`` directive is based on the values contained in ``tzname``
and whether ``daylight`` is true. Because of this, it is platform-specific
Unicode database version 3.2 instead, for applications that require this
specific version of the Unicode database (such as IDNA).
-Examples::
+Examples:
+ >>> import unicodedata
>>> unicodedata.lookup('LEFT CURLY BRACKET')
u'{'
>>> unicodedata.name('/')
smaller parts (for example, the network location is a single string), and %
escapes are not expanded. The delimiters as shown above are not part of the
result, except for a leading slash in the *path* component, which is retained if
- present. For example::
+ present. For example:
>>> from urlparse import urlparse
>>> o = urlparse('http://www.cwi.nl:80/%7Eguido/Python.html')
- >>> o
- ('http', 'www.cwi.nl:80', '/%7Eguido/Python.html', '', '', '')
+ >>> o # doctest: +NORMALIZE_WHITESPACE
+ ParseResult(scheme='http', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html',
+ params='', query='', fragment='')
>>> o.scheme
'http'
>>> o.port
Construct a full ("absolute") URL by combining a "base URL" (*base*) with
another URL (*url*). Informally, this uses components of the base URL, in
particular the addressing scheme, the network location and (part of) the path,
- to provide missing components in the relative URL. For example::
+ to provide missing components in the relative URL. For example:
>>> from urlparse import urljoin
>>> urljoin('http://www.cwi.nl/%7Eguido/Python.html', 'FAQ.html')
If *url* is an absolute URL (that is, starting with ``//`` or ``scheme://``),
the *url*'s host name and/or scheme will be present in the result. For example:
- ::
+ .. doctest::
>>> urljoin('http://www.cwi.nl/%7Eguido/Python.html',
... '//www.python.org/%7Eguido')
and fragment identifiers will be removed.
The result of this method is a fixpoint if passed back through the original
- parsing function::
+ parsing function:
>>> import urlparse
>>> url = 'HTTP://www.Python.org/doc/#'
:term:`garbage collection` is free to destroy the referent and reuse its memory
for something else. A primary use for weak references is to implement caches or
mappings holding large objects, where it's desired that a large object not be
-kept alive solely because it appears in a cache or mapping. For example, if you
-have a number of large binary image objects, you may wish to associate a name
-with each. If you used a Python dictionary to map names to images, or images to
-names, the image objects would remain alive just because they appeared as values
-or keys in the dictionaries. The :class:`WeakKeyDictionary` and
-:class:`WeakValueDictionary` classes supplied by the :mod:`weakref` module are
-an alternative, using weak references to construct mappings that don't keep
-objects alive solely because they appear in the mapping objects. If, for
-example, an image object is a value in a :class:`WeakValueDictionary`, then when
-the last remaining references to that image object are the weak references held
-by weak mappings, garbage collection can reclaim the object, and its
-corresponding entries in weak mappings are simply deleted.
+kept alive solely because it appears in a cache or mapping.
+
+For example, if you have a number of large binary image objects, you may wish to
+associate a name with each. If you used a Python dictionary to map names to
+images, or images to names, the image objects would remain alive just because
+they appeared as values or keys in the dictionaries. The
+:class:`WeakKeyDictionary` and :class:`WeakValueDictionary` classes supplied by
+the :mod:`weakref` module are an alternative, using weak references to construct
+mappings that don't keep objects alive solely because they appear in the mapping
+objects. If, for example, an image object is a value in a
+:class:`WeakValueDictionary`, then when the last remaining references to that
+image object are the weak references held by weak mappings, garbage collection
+can reclaim the object, and its corresponding entries in weak mappings are
+simply deleted.
:class:`WeakKeyDictionary` and :class:`WeakValueDictionary` use weak references
in their implementation, setting up callback functions on the weak references
directly. The low-level machinery used by the weak dictionary implementations
is exposed by the :mod:`weakref` module for the benefit of advanced uses.
+.. note::
+
+ Weak references to an object are cleared before the object's :meth:`__del__`
+ is called, to ensure that the weak reference callback (if any) finds the
+ object still alive.
+
Not all objects can be weakly referenced; those objects which can include class
instances, functions written in Python (but not in C), instance methods, sets,
frozensets, file objects, :term:`generator`\s, type objects, :class:`DBcursor`
.. note::
- Caution: Because a :class:`WeakKeyDictionary` is built on top of a Python
+ Caution: Because a :class:`WeakKeyDictionary` is built on top of a Python
dictionary, it must not change size when iterating over it. This can be
- difficult to ensure for a :class:`WeakKeyDictionary` because actions performed
- by the program during iteration may cause items in the dictionary to vanish "by
- magic" (as a side effect of garbage collection).
+ difficult to ensure for a :class:`WeakKeyDictionary` because actions
+ performed by the program during iteration may cause items in the
+ dictionary to vanish "by magic" (as a side effect of garbage collection).
:class:`WeakKeyDictionary` objects have the following additional methods. These
expose the internal references directly. The references are not guaranteed to
----------------------
Weak reference objects have no attributes or methods, but do allow the referent
-to be obtained, if it still exists, by calling it::
+to be obtained, if it still exists, by calling it:
>>> import weakref
>>> class Object:
True
If the referent no longer exists, calling the reference object returns
-:const:`None`::
+:const:`None`:
>>> del o, o2
>>> print(r())
return i
def LOWU32(i):
- """Return the low-order 32 bits of an int, as a non-negative int."""
+ """Return the low-order 32 bits, as a non-negative int"""
return i & 0xFFFFFFFF
-def write32(output, value):
- output.write(struct.pack("<l", value))
-
def write32u(output, value):
# The L format writes the bit pattern correctly whether signed
# or unsigned.
output.write(struct.pack("<L", value))
def read32(input):
- return struct.unpack("<l", input.read(4))[0]
+ return struct.unpack("<I", input.read(4))[0]
def open(filename, mode="rb", compresslevel=9):
"""Shorthand for GzipFile(filename, mode, compresslevel).
def _init_write(self, filename):
self.name = filename
- self.crc = zlib.crc32("")
+ self.crc = zlib.crc32("") & 0xffffffff
self.size = 0
self.writebuf = []
self.bufsize = 0
self.fileobj.write(fname + b'\000')
def _init_read(self):
- self.crc = zlib.crc32("")
+ self.crc = zlib.crc32("") & 0xffffffff
self.size = 0
def _read_gzip_header(self):
raise ValueError("write() on closed GzipFile object")
if len(data) > 0:
self.size = self.size + len(data)
- self.crc = zlib.crc32(data, self.crc)
+ self.crc = zlib.crc32(data, self.crc) & 0xffffffff
self.fileobj.write( self.compress.compress(data) )
self.offset += len(data)
self._new_member = True
def _add_read_data(self, data):
- self.crc = zlib.crc32(data, self.crc)
+ self.crc = zlib.crc32(data, self.crc) & 0xffffffff
self.extrabuf = self.extrabuf + data
self.extrasize = self.extrasize + len(data)
self.size = self.size + len(data)
# stored is the true file size mod 2**32.
self.fileobj.seek(-8, 1)
crc32 = read32(self.fileobj)
- isize = U32(read32(self.fileobj)) # may exceed 2GB
- if U32(crc32) != U32(self.crc):
- raise IOError("CRC check failed %s != %s" % (hex(U32(crc32)),
- hex(U32(self.crc))))
- elif isize != LOWU32(self.size):
+ isize = read32(self.fileobj) # may exceed 2GB
+ if crc32 != self.crc:
+ raise IOError("CRC check failed %s != %s" % (hex(crc32),
+ hex(self.crc)))
+ elif isize != self.size:
raise IOError("Incorrect length of data produced")
def close(self):
if self.mode == WRITE:
self.fileobj.write(self.compress.flush())
- # The native zlib crc is an unsigned 32-bit integer, but
- # the Python wrapper implicitly casts that to a signed C
- # long. So, on a 32-bit box self.crc may "look negative",
- # while the same crc on a 64-bit box may "look positive".
- # To avoid irksome warnings from the `struct` module, force
- # it to look positive on all boxes.
- write32u(self.fileobj, LOWU32(self.crc))
+ write32u(self.fileobj, self.crc)
# self.size may exceed 2GB, or even 4GB
- write32u(self.fileobj, LOWU32(self.size))
+ write32u(self.fileobj, self.size)
self.fileobj = None
elif self.mode == READ:
self.fileobj = None
distributions on the real line:
------------------------------
uniform
+ triangular
normal (Gaussian)
lognormal
negative exponential
"""
+from __future__ import division
from warnings import warn as _warn
from types import MethodType as _MethodType, BuiltinMethodType as _BuiltinMethodType
from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil
__all__ = ["Random","seed","random","uniform","randint","choice","sample",
"randrange","shuffle","normalvariate","lognormvariate",
- "expovariate","vonmisesvariate","gammavariate",
+ "expovariate","vonmisesvariate","gammavariate","triangular",
"gauss","betavariate","paretovariate","weibullvariate",
"getstate","setstate", "getrandbits",
"SystemRandom"]
"""Get a random number in the range [a, b)."""
return a + (b-a) * self.random()
+## -------------------- triangular --------------------
+
+ def triangular(self, low=0.0, high=1.0, mode=None):
+ """Triangular distribution.
+
+ Continuous distribution bounded by given lower and upper limits,
+ and having a given mode value in-between.
+
+ http://en.wikipedia.org/wiki/Triangular_distribution
+
+ """
+ u = self.random()
+ c = 0.5 if mode is None else (mode - low) / (high - low)
+ if u > c:
+ u = 1.0 - u
+ c = 1.0 - c
+ low, high = high, low
+ return low + (high - low) * (u * c) ** 0.5
+
## -------------------- normal distribution --------------------
def normalvariate(self, mu, sigma):
_test_generator(N, gammavariate, (200.0, 1.0))
_test_generator(N, gauss, (0.0, 1.0))
_test_generator(N, betavariate, (3.0, 3.0))
+ _test_generator(N, triangular, (0.0, 1.0, 1.0/3.0))
# Create one instance, seeded from current time, and export its methods
# as module-level functions. The functions share state across all uses
seed = _inst.seed
random = _inst.random
uniform = _inst.uniform
+triangular = _inst.triangular
randint = _inst.randint
choice = _inst.choice
randrange = _inst.randrange
def thfunc1(cls):
d1 = Decimal(1)
d3 = Decimal(3)
- cls.assertEqual(d1/d3, Decimal('0.333333333'))
+ test1 = d1/d3
cls.synchro.wait()
- cls.assertEqual(d1/d3, Decimal('0.333333333'))
+ test2 = d1/d3
cls.finish1.set()
+
+ cls.assertEqual(test1, Decimal('0.333333333'))
+ cls.assertEqual(test2, Decimal('0.333333333'))
return
def thfunc2(cls):
d1 = Decimal(1)
d3 = Decimal(3)
- cls.assertEqual(d1/d3, Decimal('0.333333333'))
+ test1 = d1/d3
thiscontext = getcontext()
thiscontext.prec = 18
- cls.assertEqual(d1/d3, Decimal('0.333333333333333333'))
+ test2 = d1/d3
cls.synchro.set()
cls.finish2.set()
+
+ cls.assertEqual(test1, Decimal('0.333333333'))
+ cls.assertEqual(test2, Decimal('0.333333333333333333'))
return
if not hasattr(select, "epoll"):
raise test_support.TestSkipped("test works only on Linux 2.6")
+try:
+ select.epoll()
+except IOError as e:
+ if e.errno == errno.ENOSYS:
+ raise test_support.TestSkipped("kernel doesn't support epoll()")
+
class TestEPoll(unittest.TestCase):
def setUp(self):
g.random = x[:].pop; g.gammavariate(1.0, 1.0)
g.random = x[:].pop; g.gammavariate(200.0, 1.0)
g.random = x[:].pop; g.betavariate(3.0, 3.0)
+ g.random = x[:].pop; g.triangular(0.0, 1.0, 1.0/3.0)
def test_avg_std(self):
# Use integration to test distribution average and standard deviation.
x = [i/float(N) for i in range(1,N)]
for variate, args, mu, sigmasqrd in [
(g.uniform, (1.0,10.0), (10.0+1.0)/2, (10.0-1.0)**2/12),
+ (g.triangular, (0.0, 1.0, 1.0/3.0), 4.0/9.0, 7.0/9.0/18.0),
(g.expovariate, (1.5,), 1/1.5, 1/1.5**2),
(g.paretovariate, (5.0,), 5.0/(5.0-1),
5.0/((5.0-1)**2*(5.0-2))),
self.test_files = []
def tearDown(self):
+ signal_alarm(0) # Didn't deadlock.
reap_children()
for fn in self.test_files:
except os.error:
pass
self.test_files[:] = []
- signal_alarm(0) # Didn't deadlock.
def pickaddr(self, proto):
if proto == socket.AF_INET:
self.sock.close()
def testConnectTimeout(self):
- # Test connect() timeout
- _timeout = 0.001
- self.sock.settimeout(_timeout)
-
# If we are too close to www.python.org, this test will fail.
# Pick a host that should be farther away.
if (socket.getfqdn().split('.')[-2:] == ['python', 'org'] or
socket.getfqdn().split('.')[-2:-1] == ['xs4all']):
self.addr_remote = ('tut.fi', 80)
+ # Lookup the IP address to avoid including the DNS lookup time
+ # with the connect time. This avoids failing the assertion that
+ # the timeout occurred fast enough.
+ self.addr_remote = (socket.gethostbyname(self.addr_remote[0]), 80)
+
+ # Test connect() timeout
+ _timeout = 0.001
+ self.sock.settimeout(_timeout)
+
_t1 = time.time()
self.failUnlessRaises(socket.error, self.sock.connect,
self.addr_remote)
print('def\n', file=out)
def test_ucs4(self):
- if sys.maxunicode == 0xFFFF:
- return
x = '\U00100000'
y = x.encode("raw-unicode-escape").decode("raw-unicode-escape")
self.assertEqual(x, y)
+ # FIXME
+ #y = r'\U00100000'
+ #x = y.encode("raw-unicode-escape").decode("raw-unicode-escape")
+ #self.assertEqual(x, y)
+ #y = r'\U00010000'
+ #x = y.encode("raw-unicode-escape").decode("raw-unicode-escape")
+ #self.assertEqual(x, y)
+
+ #try:
+ # '\U11111111'.decode("raw-unicode-escape")
+ #except UnicodeDecodeError as e:
+ # self.assertEqual(e.start, 0)
+ # self.assertEqual(e.end, 10)
+ #else:
+ # self.fail("Should have raised UnicodeDecodeError")
+
def test_conversion(self):
# Make sure __unicode__() works properly
class Foo0:
# protocol error; provide additional information in test output
self.fail("%s\n%s" % (e, getattr(e, "headers", "")))
+ def test_dotted_attribute(self):
+ # Raises an AttributeError because private methods are not allowed.
+ self.assertRaises(AttributeError,
+ SimpleXMLRPCServer.resolve_dotted_attribute, str, '__add')
+
+ self.assert_(SimpleXMLRPCServer.resolve_dotted_attribute(str, 'title'))
+ # Get the test to run faster by sending a request with test_simple1.
+ # This avoids waiting for the socket timeout.
+ self.test_simple1()
+
# This is a contrived way to make a failure occur on the server side
# in order to test the _send_traceback_header flag on the server
class FailingMessageClass(mimetools.Message):
if (*f == '%') {
const char* p = f;
width = 0;
- while (ISDIGIT(*f))
+ while (ISDIGIT((unsigned)*f))
width = (width*10) + *f++ - '0';
- while (*++f && *f != '%' && !ISALPHA(*f))
+ while (*++f && *f != '%' && !ISALPHA((unsigned)*f))
;
/* skip the 'l' or 'z' in {%ld, %zd, %lu, %zu} since
zeropad = (*f == '0');
/* parse the width.precision part */
width = 0;
- while (ISDIGIT(*f))
+ while (ISDIGIT((unsigned)*f))
width = (width*10) + *f++ - '0';
precision = 0;
if (*f == '.') {
f++;
- while (ISDIGIT(*f))
+ while (ISDIGIT((unsigned)*f))
precision = (precision*10) + *f++ - '0';
}
/* handle the long flag, but only for %ld and %lu.
else
x += 10 + c - 'A';
}
-#ifndef Py_UNICODE_WIDE
- if (x > 0x10000) {
+ if (x <= 0xffff)
+ /* UCS-2 character */
+ *p++ = (Py_UNICODE) x;
+ else if (x <= 0x10ffff) {
+ /* UCS-4 character. Either store directly, or as
+ surrogate pair. */
+#ifdef Py_UNICODE_WIDE
+ *p++ = (Py_UNIC0DE) x;
+#else
+ x -= 0x10000L;
+ *p++ = 0xD800 + (Py_UNICODE) (x >> 10);
+ *p++ = 0xDC00 + (Py_UNICODE) (x & 0x03FF);
+#endif
+ } else {
+ endinpos = s-starts;
+ outpos = p-PyUnicode_AS_UNICODE(v);
if (unicode_decode_call_errorhandler(
errors, &errorHandler,
"rawunicodeescape", "\\Uxxxxxxxx out of range",
(PyObject **)&v, &outpos, &p))
goto onError;
}
-#endif
- *p++ = x;
nextByte:
;
}
*p++ = hexdigits[ch & 15];
}
else
+#else
+ /* Map UTF-16 surrogate pairs to '\U00xxxxxx' */
+ if (ch >= 0xD800 && ch < 0xDC00) {
+ Py_UNICODE ch2;
+ Py_UCS4 ucs;
+
+ ch2 = *s++;
+ size--;
+ if (ch2 >= 0xDC00 && ch2 <= 0xDFFF) {
+ ucs = (((ch & 0x03FF) << 10) | (ch2 & 0x03FF)) + 0x00010000;
+ *p++ = '\\';
+ *p++ = 'U';
+ *p++ = hexdigits[(ucs >> 28) & 0xf];
+ *p++ = hexdigits[(ucs >> 24) & 0xf];
+ *p++ = hexdigits[(ucs >> 20) & 0xf];
+ *p++ = hexdigits[(ucs >> 16) & 0xf];
+ *p++ = hexdigits[(ucs >> 12) & 0xf];
+ *p++ = hexdigits[(ucs >> 8) & 0xf];
+ *p++ = hexdigits[(ucs >> 4) & 0xf];
+ *p++ = hexdigits[ucs & 0xf];
+ continue;
+ }
+ /* Fall through: isolated surrogates are copied as-is */
+ s--;
+ size++;
+ }
#endif
/* Map 16-bit characters to '\uxxxx' */
if (ch >= 256) {
-/* Cross platform case insenstive string compare functions
+/* Cross platform case insensitive string compare functions
*/
#include "Python.h"
{
if (size == 0)
return 0;
- while ((--size > 0) && (tolower(*s1) == tolower(*s2))) {
+ while ((--size > 0) &&
+ (tolower((unsigned)*s1) == tolower((unsigned)*s2))) {
if (!*s1++ || !*s2++)
break;
}
- return tolower(*s1) - tolower(*s2);
+ return tolower((unsigned)*s1) - tolower((unsigned)*s2);
}
int
PyOS_mystricmp(const char *s1, const char *s2)
{
- while (*s1 && (tolower(*s1++) == tolower(*s2++))) {
+ while (*s1 && (tolower((unsigned)*s1++) == tolower((unsigned)*s2++))) {
;
}
- return (tolower(*s1) - tolower(*s2));
+ return (tolower((unsigned)*s1) - tolower((unsigned)*s2));
}
libraries=math_libs) )
exts.append( Extension('datetime', ['datetimemodule.c', 'timemodule.c'],
libraries=math_libs) )
- # random number generator implemented in C
- exts.append( Extension("_random", ["_randommodule.c"]) )
# fast iterator tools implemented in C
exts.append( Extension("itertools", ["itertoolsmodule.c"]) )
+ # random number generator implemented in C
+ exts.append( Extension("_random", ["_randommodule.c"]) )
# high-performance collections
exts.append( Extension("_collections", ["_collectionsmodule.c"]) )
# bisect
projects / thirdparty / Python / cpython.git / commitdiff
