What’s New In Python 3.4

This article explains the new features in Python 3.4, compared to 3.3.

For full details, see the changelog.

Note

Prerelease users should be aware that this document is currently in draft form. It will be updated substantially as Python 3.4 moves towards release, so it’s worth checking back even after reading earlier versions.

See also

Summary – Release highlights

New syntax features:

  • None yet.

New library modules:

  • enum: Implementation of the PEP 435.
  • selectors: High-level and efficient I/O multiplexing, built upon the select module primitives.

New built-in features:

Implementation improvements:

  • A more efficient marshal format (issue 16475).
  • Improve finalization of Python modules to avoid setting their globals to None, in most cases (issue 18214).

Significantly Improved Library Modules:

Security improvements:

Please read on for a comprehensive list of user-facing changes.

PEP 446: Make newly created file descriptors non-inheritable

The PEP 446 makes newly created file descriptors non-inheritable. New functions and methods:

PEP 445: Add new APIs to customize Python memory allocators

The PEP 445 adds new Application Programming Interfaces (API) to customize Python memory allocators.

PEP 442: Safe object finalization

This PEP removes the current limitations and quirks of object finalization. With it, objects with __del__() methods, as well as generators with finally clauses, can be finalized when they are part of a reference cycle.

As part of this change, module globals are no longer forcibly set to None during interpreter shutdown, instead relying on the normal operation of the cyclic garbage collector.

See also

PEP 442 - Safe object finalization
PEP written and implemented by Antoine Pitrou

Other Language Changes

Some smaller changes made to the core Python language are:

  • Unicode database updated to UCD version 6.2.
  • min() and max() now accept a default argument that can be used to specify the value they return if the iterable they are evaluating has no elements. Contributed by Julian Berman in issue 18111.
  • Module objects are now weakref‘able.

New Modules

selectors

The new selectors module allows high-level and efficient I/O multiplexing, built upon the select module primitives.

Improved Modules

aifc

The getparams() method now returns a namedtuple rather than a plain tuple. (Contributed by Claudiu Popa in issue 17818.)

colorsys

The number of digits in the coefficients for the RGB — YIQ conversions have been expanded so that they match the FCC NTSC versions. The change in results should be less than 1% and may better match results found elsewhere.

dis

The dis module is now built around an Instruction class that provides details of individual bytecode operations and a get_instructions() iterator that emits the Instruction stream for a given piece of Python code. The various display tools in the dis module have been updated to be based on these new components.

The new dis.Bytecode class provides an object-oriented API for inspecting bytecode, both in human-readable form and for iterating over instructions.

(Contributed by Nick Coghlan, Ryan Kelly and Thomas Kluyver in issue 11816)

doctest

Added FAIL_FAST flag to halt test running as soon as the first failure is detected. (Contributed by R. David Murray and Daniel Urban in issue 16522.)

Updated the doctest command line interface to use argparse, and added -o and -f options to the interface. -o allows doctest options to be specified on the command line, and -f is a shorthand for -o FAIL_FAST (to parallel the similar option supported by the unittest CLI). (Contributed by R. David Murray in issue 11390.)

email

as_string() now accepts a policy argument to override the default policy of the message when generating a string representation of it. This means that as_string can now be used in more circumstances, instead of having to create and use a generator in order to pass formatting parameters to its flatten method.

New method as_bytes() added to produce a bytes representation of the message in a fashion similar to how as_string produces a string representation. It does not accept the maxheaderlen argument, but does accept the unixfrom and policy arguments. The Message __bytes__() method calls it, meaning that bytes(mymsg) will now produce the intuitive result: a bytes object containing the fully formatted message.

(Contributed by R. David Murray in issue 18600.)

functools

New functools.singledispatch() decorator: see the PEP 443.

inspect

The inspect module now offers a basic command line interface to quickly display source code and other information for modules, classes and functions.

unwrap() makes it easy to unravel wrapper function chains created by functools.wraps() (and any other API that sets the __wrapped__ attribute on a wrapper function).

mmap

mmap objects can now be weakref’ed.

(Contributed by Valerie Lambert in issue 4885.)

multiprocessing

On Unix two new start methods have been added for starting processes using multiprocessing. These make the mixing of processes with threads more robust. See issue 8713.

Also, except when using the old fork start method, child processes will no longer inherit unneeded handles/file descriptors from their parents.

os

New functions to get and set the inheritable flag of a file descriptors or a Windows handle:

pdb

The print command has been removed from pdb, restoring access to the print function.

Rationale: Python2’s pdb did not have a print command; instead, entering print executed the print statement. In Python3 print was mistakenly made an alias for the pdb p command. p, however, prints the repr of its argument, not the str like the Python2 print command did. Worse, the Python3 pdb print command shadowed the Python3 print function, making it inaccessible at the pdb prompt.

(Contributed by Connor Osborn in issue 18764.)

poplib

New stls() method to switch a clear-text POP3 session into an encrypted POP3 session.

New capa() method to query the capabilities advertised by the POP3 server.

(Contributed by Lorenzo Catucci in issue 4473.)

pprint

The :mod::pprint module now supports compact mode for formatting long sequences (issue 19132).

smtplib

SMTPException is now a subclass of OSError, which allows both socket level errors and SMTP protocol level errors to be caught in one try/except statement by code that only cares whether or not an error occurred. (issue 2118).

socket

Socket objects have new methods to get or set their inheritable flag:

The socket.AF_* and socket.SOCK_* constants are enumeration values, using the new enum module. This allows descriptive reporting during debugging, instead of seeing integer “magic numbers”.

ssl

TLSv1.1 and TLSv1.2 support.

(Contributed by Michele Orrù and Antoine Pitrou in issue 16692)

(Contributed by Christian Heimes in issue 18143, issue 18147 and
issue 17134.)

Support for server-side SNI using the new ssl.SSLContext.set_servername_callback() method.

(Contributed by Daniel Black in issue 8109.)

stat

The stat module is now backed by a C implementation in _stat. A C implementation is required as most of the values aren’t standardized and platform-dependent. (Contributed by Christian Heimes in issue 11016.)

The module supports new file types: door, event port and whiteout.

struct

Streaming struct unpacking using struct.iter_unpack().

(Contributed by Antoine Pitrou in issue 17804.)

sunau

The getparams() method now returns a namedtuple rather than a plain tuple. (Contributed by Claudiu Popa in issue 18901.)

sunau.open() now supports the context manager protocol (issue 18878).

traceback

A new traceback.clear_frames() function takes a traceback object and clears the local variables in all of the frames it references, reducing the amount of memory consumed (issue 1565525).

urllib

Add support.for data: URLs in urllib.request.

(Contributed by Mathias Panzenböck in issue 16423.)

unittest

Support for easy dynamically-generated subtests using the subTest() context manager.

(Contributed by Antoine Pitrou in issue 16997.)

wave

The getparams() method now returns a namedtuple rather than a plain tuple. (Contributed by Claudiu Popa in issue 17487.)

wave.open() now supports the context manager protocol. (Contributed by Claudiu Popa in issue 17616.)

weakref

New WeakMethod class simulates weak references to bound methods. (Contributed by Antoine Pitrou in issue 14631.)

New finalize class makes it possible to register a callback to be invoked when an object is garbage collected, without needing to carefully manage the lifecycle of the weak reference itself. (Contributed by Richard Oudkerk in issue 15528)

xml.etree

Add an event-driven parser for non-blocking applications, XMLPullParser.

(Contributed by Antoine Pitrou in issue 17741.)

Other improvements

Tab-completion is now enabled by default in the interactive interpreter.

(Contributed by Antoine Pitrou and Éric Araujo in issue 5845.)

Python invocation changes

Invoking the Python interpreter with --version now outputs the version to standard output instead of standard error (issue 18338). Similar changes were made to argparse (issue 18920) and other modules that have script-like invocation capabilities (issue 18922).

Optimizations

Major performance enhancements have been added:

  • The UTF-32 decoder is now 3x to 4x faster.

  • The cost of hash collisions for sets is now reduced. Each hash table probe now checks a series of consecutive, adjacent key/hash pairs before continuing to make random probes through the hash table. This exploits cache locality to make collision resolution less expensive.

    The collision resolution scheme can be described as a hybrid of linear probing and open addressing. The number of additional linear probes defaults to nine. This can be changed at compile-time by defining LINEAR_PROBES to be any value. Set LINEAR_PROBES=0 to turn-off linear probing entirely.

    (Contributed by Raymond Hettinger in :issue”18771.)

Build and C API Changes

Changes to Python’s build process and to the C API include:

  • None yet.

Deprecated

Unsupported Operating Systems

  • OS/2
  • Windows 2000

Deprecated Python modules, functions and methods

Deprecated functions and types of the C API

  • The PyThreadState.tick_counter field has been value: its value was meaningless since Python 3.2 (“new GIL”).

Deprecated features

  • None yet.

Porting to Python 3.4

This section lists previously described changes and other bugfixes that may require changes to your code.

  • The ABCs defined in importlib.abc now either raise the appropriate exception or return a default value instead of raising NotImplementedError blindly. This will only affect code calling super() and falling through all the way to the ABCs. For compatibility, catch both NotImplementedError or the appropriate exception as needed.
  • The module type now initializes the __package__ and __loader__ attributes to None by default. To determine if these attributes were set in a backwards-compatible fashion, use e.g. getattr(module, '__loader__', None) is not None.
  • importlib.util.module_for_loader() now sets __loader__ and __package__ unconditionally to properly support reloading. If this is not desired then you will need to set these attributes manually. You can use importlib.util.module_to_load() for module management.
  • Import now resets relevant attributes (e.g. __name__, __loader__, __package__, __file__, __cached__) unconditionally when reloading.
  • Frozen packages no longer set __path__ to a list containing the package name but an empty list instead. Determing if a module is a package should be done using hasattr(module, '__path__').
  • PyErr_SetImportError() now sets TypeError when its msg argument is not set. Previously only NULL was returned with no exception set.
  • py_compile.compile() now raises FileExistsError if the file path it would write to is a symlink or a non-regular file. This is to act as a warning that import will overwrite those files with a regular file regardless of what type of file path they were originally.
  • importlib.abc.SourceLoader.get_source() no longer raises ImportError when the source code being loaded triggers a SyntaxError or UnicodeDecodeError. As ImportError is meant to be raised only when source code cannot be found but it should, it was felt to be over-reaching/overloading of that meaning when the source code is found but improperly structured. If you were catching ImportError before and wish to continue to ignore syntax or decoding issues, catch all three exceptions now.
  • functools.update_wrapper() and functools.wraps() now correctly set the __wrapped__ attribute even if the wrapped function had a wrapped attribute set. This means __wrapped__ attributes now correctly link a stack of decorated functions rather than every __wrapped__ attribute in the chain referring to the innermost function. Introspection libraries that assumed the previous behaviour was intentional can use inspect.unwrap() to gain equivalent behaviour.
  • (C API) The result of the :c:var:`PyOS_ReadlineFunctionPointer` callback must now be a string allocated by PyMem_RawMalloc() or PyMem_RawRealloc(), or NULL if an error occurred, instead of a string allocated by PyMem_Malloc() or PyMem_Realloc().