During Icehouse release cycle in order to drop dependency on eventlet we removed eventlet tpool.Proxy helper and the corresponding config option (use_tpool) from oslo.db code. Projects were supposed to store those in their source trees. Since then we’ve got a lot of push back from Nova and Cinder teams based on their experience of adoption of oslo.db changes. To make things easier for oslo.db users, it’d actually be better to provide optional integration with eventlet tpool.Proxy as a separate module within oslo.db tree.
Most OpenStack deployments nowadays use MySQL. Unfortunately, the most popular MySQL Python DBAPI driver - MySQL-Python - has a serious problem with eventlet green threads: being a C-extension it hangs the process on blocking DB queries (as eventlet can’t monkey patch it to force a green thread switch on blocking reads/writes from/to a socket).
eventlet provides a work around for this problem: tpool.Proxy helper class is meant to wrap classes/modules which methods/functions might possibly block the execution of a green thread. The call is performed in the context of a real OS thread, which knows how to deal with eventlet thread pool, on return it will switch the execution context back to a green thread, so the process doesn’t hang.
Common DB code used to provide use_tpool option which enabled tpool.Proxy proxying for all database methods calls. This was used by Nova at least. It’s worth mentioning, that in order to actually use use_tpool option one had to use a patched version of eventlet, as neither PyPI releases nor master HEAD play nicely with MySQL-Python.
During Icehouse release cycle we worked on removing extra dependencies from common DB code. eventlet was removed as one of the dependencies we thought were unnecessary (as oslo.db should neither enforce you to use a particular concurrency model, nor it can make any assumptions how the target projects process concurrent requests, whether they use eventlet green threads, real OS threads, multiple processing, etc). But the best is the enemy of the good, and it seems that it’s actually better to provide optional integration with eventlet tpool.Proxy, so the people wouldn’t need to do this in their source trees.
To make it possible to use oslo.db with MySQL-Python DB API driver and eventlet tpool.Proxy call proxying we need:
The alternative way to solve this would be:
The advantage of this solution is that we don’t need to re-add eventlet stuff back to oslo.db.
The disadvantage of this solution is that we add more interdependencies between oslo.* libraries and make it harder to use oslo.db (it’d probably be not very clear to users why they need to use another library, if they use eventlet and MySQL together).
Enabling of use_tpool improves performance of DB-bound services (APIs) a lot, when eventlet and MySQL-Python are used. Note: you will still need to use unreleased eventlet code.
use_tpool won’t be a new config file option, we just introduce it in the context of oslo.db. Currently, projects consume it from openstack.common.db package. No changes will be needed from deployers perspective.
OpenStack projects that want to use oslo.db with eventlet tpool.Proxy call proxying will need to switch from using of DBAPI class to the new helper.
use_tpool should be marked as experimental. It should be stated in docs, that patched version of eventlet is needed to use it.
To work properly, this depends on the unreleased version of eventlet. Though, we’ve been providing use_tpool option so far anyway. Rackspace uses patched version of eventlet in production. We are going to leave it up to deployers to decide whether to enable this or not (default was and remains False).
A PoC patch on review:
The related ML thread:
The Launchpad bug for tracking usage of use_tpool in projects:
eventlet tpool docs:
The patch fixing the eventlet issue:
This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode