diff --git a/doc/changelog.rst b/doc/changelog.rst index 8c4d5801e..b55fae212 100644 --- a/doc/changelog.rst +++ b/doc/changelog.rst @@ -17,6 +17,7 @@ Highlights include: `tzinfo` option of :class:`~bson.codec_options.CodecOptions`. - An implementation of :class:`~gridfs.GridFSBucket` from the new GridFS spec. - Compliance with the new Connection String spec. +- Reduced idle CPU usage in Python 2. Changes in internal classes ........................... diff --git a/doc/developer/periodic_executor.rst b/doc/developer/periodic_executor.rst index b794ad80a..6327cfd83 100644 --- a/doc/developer/periodic_executor.rst +++ b/doc/developer/periodic_executor.rst @@ -7,22 +7,6 @@ PyMongo implements a :class:`~periodic_executor.PeriodicExecutor` for two purposes: as the background thread for :class:`~monitor.Monitor`, and to regularly check if there are `OP_KILL_CURSORS` messages that must be sent to the server. -Monitoring ----------- - -For each server in the topology, :class:`~topology.Topology` launches a -monitor thread. This thread must not prevent the topology from being freed, -so it weakrefs the topology. Furthermore, it uses a weakref callback to close -itself promptly when the topology is freed. - -Solid lines represent strong references, dashed lines weak ones: - -.. generated with graphviz from periodic-executor-refs.dot - -.. image:: ../static/periodic-executor-refs.png - -See `Stopping Executors`_ below for an explanation of ``_EXECUTORS``. - Killing Cursors --------------- @@ -35,16 +19,17 @@ the cursor before finishing iteration:: We try to send an `OP_KILL_CURSORS` to the server to tell it to clean up the server-side cursor. But we must not take any locks directly from the cursor's -destructor (see `PYTHON-799 `_), -so we cannot safely use the PyMongo data structures required to send a message. -The solution is to add the cursor's id to an array on the -:class:`~mongo_client.MongoClient` without taking any locks. +destructor (see `PYTHON-799`_), so we cannot safely use the PyMongo data +structures required to send a message. The solution is to add the cursor's id +to an array on the :class:`~mongo_client.MongoClient` without taking any locks. Each client has a :class:`~periodic_executor.PeriodicExecutor` devoted to checking the array for cursor ids. Any it sees are the result of cursors that were freed while the server-side cursor was still open. The executor can safely take the locks it needs in order to send the `OP_KILL_CURSORS` message. +.. _PYTHON-799: https://jira.mongodb.org/browse/PYTHON-799 + Stopping Executors ------------------ @@ -55,7 +40,7 @@ the topology calls :meth:`close` on all its monitor threads, the :meth:`close` method cannot actually call :meth:`wake` on the executor, since :meth:`wake` takes a lock. -Instead, executors wake very frequently to check if ``self.close`` is set, +Instead, executors wake periodically to check if ``self.close`` is set, and if so they exit. A thread can log spurious errors if it wakes late in the Python interpreter's @@ -67,3 +52,62 @@ An `exit handler`_ runs on shutdown and tells all executors to stop, then tries (with a short timeout) to join all executor threads. .. _exit handler: https://docs.python.org/2/library/atexit.html + +Monitoring +---------- + +For each server in the topology, :class:`~topology.Topology` uses a periodic +executor to launch a monitor thread. This thread must not prevent the topology +from being freed, so it weakrefs the topology. Furthermore, it uses a weakref +callback to terminate itself soon after the topology is freed. + +Solid lines represent strong references, dashed lines weak ones: + +.. generated with graphviz: "dot -Tpng periodic-executor-refs.dot > periodic-executor-refs.png" + +.. image:: ../static/periodic-executor-refs.png + +See `Stopping Executors`_ above for an explanation of the ``_EXECUTORS`` set. + +It is a requirement of the `Server Discovery And Monitoring Spec`_ that a +sleeping monitor can be awakened early. Aside from infrequent wakeups to do +their appointed chores, and occasional interruptions, periodic executors also +wake periodically to check if they should terminate. + +Our first implementation of this idea was the obvious one: use the Python +standard library's threading.Condition.wait with a timeout. Another thread +wakes the executor early by signaling the condition variable. + +A topology cannot signal the condition variable to tell the executor to +terminate, because it would risk a deadlock in the garbage collector: no +destructor or weakref callback can take a lock to signal the condition variable +(see `PYTHON-863`_); thus the only way for a dying object to terminate a +periodic executor is to set its "stopped" flag and let the executor see the +flag next time it wakes. + +We erred on the side of prompt cleanup, and set the check interval at 100ms. We +assumed that checking a flag and going back to sleep 10 times a second was +cheap on modern machines. + +Starting in Python 3.2, the builtin C implementation of lock.acquire takes a +timeout parameter, so Python 3.2+ Condition variables sleep simply by calling +lock.acquire; they are implemented as efficiently as expected. + +But in Python 2, lock.acquire has no timeout. To wait with a timeout, a Python +2 condition variable sleeps a millisecond, tries to acquire the lock, sleeps +twice as long, and tries again. This exponential backoff reaches a maximum +sleep time of 50ms. + +If PyMongo calls the condition variable's "wait" method with a short timeout, +the exponential backoff is restarted frequently. Overall, the condition variable +is not waking a few times a second, but hundreds of times. (See `PYTHON-983`_.) + +Thus the current design of periodic executors is surprisingly simple: they +do a simple `time.sleep` for a half-second, check if it is time to wake or +terminate, and sleep again. + +.. _Server Discovery And Monitoring Spec: https://github.com/mongodb/specifications/blob/master/source/server-discovery-and-monitoring/server-discovery-and-monitoring.rst#requesting-an-immediate-check + +.. _PYTHON-863: https://jira.mongodb.org/browse/PYTHON-863 + +.. _PYTHON-983: https://jira.mongodb.org/browse/PYTHON-983 diff --git a/doc/static/periodic-executor-refs.dot b/doc/static/periodic-executor-refs.dot index 003af9f57..39632cfa7 100644 --- a/doc/static/periodic-executor-refs.dot +++ b/doc/static/periodic-executor-refs.dot @@ -5,6 +5,7 @@ digraph "Monitor and PeriodicExecutor" { monitor -> executor executor -> "target()" "target()" -> self_ref + thread -> "target()" // Weak references edge [style="dashed"]; diff --git a/doc/static/periodic-executor-refs.png b/doc/static/periodic-executor-refs.png index 9530a5669..3224b3d1d 100644 Binary files a/doc/static/periodic-executor-refs.png and b/doc/static/periodic-executor-refs.png differ diff --git a/pymongo/periodic_executor.py b/pymongo/periodic_executor.py index 384b6063e..8b7015b3f 100644 --- a/pymongo/periodic_executor.py +++ b/pymongo/periodic_executor.py @@ -37,6 +37,8 @@ class PeriodicExecutor(object): """ # threading.Event and its internal condition variable are expensive # in Python 2, see PYTHON-983. Use a boolean to know when to wake. + # The executor's design is constrained by several Python issues, see + # "periodic_executor.rst" in this repository. self._event = False self._interval = interval self._min_interval = min_interval