Adding Python 3.4 support to Nova

https://blueprints.launchpad.net/nova/+spec/nova-python3

Problem description

It’s time to add Python 3 support to Nova by generalizing the usage of the six module, in addition to the Python 2 support.

Use Cases

See the article Why should OpenStack move to Python 3 right now? for the rationale.

Project Priority

None

Proposed change

This specification details the steps needed to add Python 3.4 support to Nova by generalizing the usage of the six module.

Almost all Nova dependencies are ported to Python 3. For the few remaining libraries, the port is in progress and should be done in a few weeks. It is already possible to start porting Nova to Python 3. See the Dependencies section below for more information.

The goal here is to make all Nova tests pass with Python 3: Nova unit tests and Tempest tests.

Note

Python 2 support is kept.

Alternatives

None

Data model impact

None (changes must not impact the data model).

REST API impact

None (changes must not impact the REST API).

Security impact

None

Notifications impact

None

Other end user impact

None

Performance Impact

There is no impact on performances.

In March 2013, Brett Canon ran the official Python benchmarks suite to compare Python 2.7 and 3.3 for his talk at Pycon US: Python 3.3: Trust Me, It’s Better Than Python 2.7. The result: “If you sorted all of the benchmarks and looked at the median result … Python 3 is the same”.

See the “Optimizations” section of each “What’s New in Python 3.x” document for the full list of optimizations: Python 3.1, Python 3.2, Python 3.3 and Python 3.4.

Other deployer impact

Deployers using python 2.7 will see no changes.

Those able to run python 3.4 will now be try that out.

Developer impact

Once the Python 3 check job becomes voting, developers will have to write code compatible with Python 2 and Python 3. During the transition period, only code tested by the tox.ini whitelists will require Python 3 support.

Thanks to tox, it is trivial to run locally the Nova test suite on Python 2.7 and 3.4.

Implementation

Assignee(s)

Primary assignee:

victor-stinner

Other contributors:

None

Work Items

  • Fix most obvious Python 3 issues. Example of a patch fixing most Python 3 issues: Port Nova to Python 3. This change is just a draft to test if porting Nova is feasible, it should be splitted into smaller patches grouped by similar changes. See also the Port Python 2 code to Python 3 section of the Python 3 wiki page.

  • Add a Python 3 test environment to tox.ini to run a subtest of the tests which pass on Python 3. It will use a whitelist of tests which are know to pass on Python 3.

  • Add a non-voting Python 3 check job for Nova

  • When the Python 3 check job is stable enough, make it voting. From this point, we should now be able to avoid Python 3 regressions while working.

  • Fix failing tests, one by one, to enlarge the whitelist of tests in tox.ini.

  • Once all tests work, remove the whitelist of tests from tox.ini.

The transition period, when Python 3 is only supported partially, should be a short as possible.

No voting Python 3 gate jobs will be added to not waste resources of the OpenStack infra, Python 2 gate jobs are enough. We consider that there is a low risk of having a Python 3 specific issue introduced by a conflict between the Python 3 check job and the Python 2 gate job.

Dependencies

Remaining dependencies not compatible with Python 3 yet:

  • oslo.messaging: The development version works on Python 3, except of Qpid and AMQP 1.0 drivers. To begin the Nova port to Python 3, we can start with the RabbitMQ driver, until AMQP is ported to Python 3 too. A new version of oslo.messaging will be released in a few weeks.

  • mysql-python: the fork mysqlclient works on Python 3 and includes bug fixes. There is also PyMySQL, a driver fully implemented in Python which works on Python 3 too, but it has worse performances.

  • python-memcached: see the pull request Port memcache to Python 3. It blocks keystonemiddleware. It may be replaced with pymemcache which is already Python 3 compatible.

  • websockify: Python 3 classifier is missing in websockify 0.6.0, but it is present in the development version. Tests are failing but they may be issues with the tests, not with websockify directly.

All in all, there is no major issue with the dependencies.

Using the development version of oslo.messaging, it’s already possible to work on fixing Nova tests on Python 3.

Testing

The current test suite should be enough to test Nova on Python 3.

We will run tests with Nova running under Python 3.4 by the end of this process: Nova unit tests and Tempest tests.

Documentation Impact

Developers might be interested in reading the official Python 3 page on the Openstack wiki. It shows the current progress of the OpenStack port of Python 3, and details some common issues that arise when porting code from Python 2 to Python 3.

References

History

Revisions

Release Name

Description

Liberty

Introduced