mongodb/mongo-python-driver
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

PYTHON-784 Deprecate start_request.

Browse Source
This commit is contained in:
A. Jesse Jiryu Davis 2014-11-06 22:18:04 -05:00
parent 4579303838
commit 0f1f99b52b
5 changed files with 55 additions and 183 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

112
doc/examples/requests.rst
View File

@ -1,11 +1,19 @@
Requests
========
PyMongo supports the idea of a *request*: a series of operations executed with
a single socket, which are guaranteed to be processed on the server in the same
order as they ran on the client.
The ``start_request`` method of :class:`~pymongo.mongo_client.MongoClient`
and :class:`~pymongo.mongo_replica_set_client.MongoReplicaSetClient` is now
**deprecated** and will be removed in PyMongo 3.0.
Requests are not usually necessary with PyMongo.
PyMongo versions previous to 3.0 support the idea of a *request*: a series of
operations executed with a single socket. This feature intended to make
read-your-writes consistency more likely, even with unacknowledged writes.
(That is, operations with write concern ``w=0``.)
However, mongos 2.6 doesn't support socket pinning by default, and `mongos 2.8
doesn't support it at all`_, so requests provide no benefit with sharding.
In any case, requests are no longer necessary with PyMongo.
By default, the methods :meth:`~pymongo.collection.Collection.insert`,
:meth:`~pymongo.collection.Collection.update`,
:meth:`~pymongo.collection.Collection.save`, and
@ -14,97 +22,7 @@ acknowledgment from the server, so ordered execution is already guaranteed. You
can be certain the next :meth:`~pymongo.collection.Collection.find` or
:meth:`~pymongo.collection.Collection.count`, for example, is executed on the
server after the writes complete. This is called "read-your-writes
consistency."
consistency." If your application requires this consistency, do not override
the default write concern with ``w=0``.
An example of when a request is necessary is if a series of documents are
inserted with ``w=0`` for performance reasons, and you want to query those
documents immediately afterward: With ``w=0`` the writes can queue up at the
server and might not be immediately visible in query results. Wrapping the
inserts and queries within
:meth:`~pymongo.mongo_client.MongoClient.start_request` and
:meth:`~pymongo.mongo_client.MongoClient.end_request` forces a query to be on
the same socket as the inserts, so the query won't execute until the inserts
are complete on the server side.
Example
-------
Let's consider a collection of web-analytics counters. We want to count the
number of page views our site has served for each combination of browser,
region, and OS, and then show the user the number of page views from his or her
region, *including* the user's own visit. We have three ways to do so reliably:
1. Simply update the counters with an acknowledged write (the default), and
then ``find`` all counters for the visitor's region. This will ensure that the
``update`` completes before the ``find`` begins, but it comes with a performance
penalty that may be unacceptable for analytics.
2. Create the :class:`~pymongo.mongo_client.MongoClient` with ``w=0`` and
``auto_start_request=True`` to do unacknowledged writes and ensure each thread
gets its own socket.
3. Explicitly call :meth:`~pymongo.mongo_client.MongoClient.start_request`,
then do the unacknowledged updates and the queries within the request. This
third method looks like:
.. testsetup::
from pymongo import MongoClient
client = MongoClient()
counts = client.requests_example.counts
counts.drop()
region, browser, os = 'US', 'Firefox', 'Mac OS X'
.. doctest::
>>> client = MongoClient()
>>> counts = client.requests_example.counts
>>> region, browser, os = 'US', 'Firefox', 'Mac OS X'
>>> request = client.start_request()
>>> try:
... counts.update(
... {'region': region, 'browser': browser, 'os': os},
... {'$inc': {'n': 1 }},
... upsert=True,
... w=0) # unacknowledged write
...
... # This always runs after update has completed:
... count = sum([p['n'] for p in counts.find({'region': region})])
... finally:
... request.end()
>>> print count
1
Requests can also be used as context managers, with the `with statement
<http://docs.python.org/reference/compound_stmts.html#index-15>`_, which makes
the previous example more terse:
.. doctest::
>>> client.in_request()
False
>>> with client.start_request():
... # MongoClient is now in request
... counts.update(
... {'region': region, 'browser': browser, 'os': os},
... {'$inc': {'n': 1 }},
... upsert=True,
... safe=False)
... print sum([p['n'] for p in counts.find({'region': region})])
2
>>> client.in_request() # request automatically ended
False
Requests And ``max_pool_size``
------------------------------
A thread in a request retains exclusive access to a socket until its request
ends or the thread dies; thus, applications in which more than 100 threads are
in requests at once should disable the ``max_pool_size`` option::
client = MongoClient(host, port, max_pool_size=None)
Failure to increase or disable ``max_pool_size`` in such an application can
leave threads forever waiting for sockets.
See :ref:`connection-pooling`
.. _mongos 2.8 doesn't support it at all: https://jira.mongodb.org/browse/SERVER-12273

3
pymongo/connection.py
View File

@ -116,8 +116,7 @@ class Connection(MongoClient):
keep-alive packets).
- `auto_start_request`: If ``True`` (the default), each thread that
accesses this Connection has a socket allocated to it for the
thread's lifetime. This ensures consistent reads, even if you read
after an unsafe write.
thread's lifetime, or until :meth:`end_request` is called.
- `use_greenlets`: if ``True``, :meth:`start_request()` will ensure
that the current greenlet uses the same socket for all operations
until :meth:`end_request()`. Defaults to ``False``.

47
pymongo/mongo_client.py
View File

@ -152,10 +152,7 @@ class MongoClient(common.BaseObject):
- `socketKeepAlive`: (boolean) Whether to send periodic keep-alive
packets on connected sockets. Defaults to ``False`` (do not send
keep-alive packets).
- `auto_start_request`: If ``True``, each thread that accesses
this :class:`MongoClient` has a socket allocated to it for the
thread's lifetime. This ensures consistent reads, even if you
read after an unacknowledged write. Defaults to ``False``.
- `auto_start_request`: Deprecated.
- `use_greenlets`: If ``True``, :meth:`start_request()` will ensure
that the current greenlet uses the same socket for all
operations until :meth:`end_request()`. Defaults to ``False``.
@ -604,8 +601,7 @@ class MongoClient(common.BaseObject):
@property
def auto_start_request(self):
"""Is auto_start_request enabled?
"""
"""**DEPRECATED** Is auto_start_request enabled?"""
return self.__auto_start_request
def get_document_class(self):
@ -1236,26 +1232,17 @@ class MongoClient(common.BaseObject):
raise AutoReconnect(str(e))
def start_request(self):
"""Ensure the current thread or greenlet always uses the same socket
until it calls :meth:`end_request`. This ensures consistent reads,
even if you read after an unacknowledged write.
"""DEPRECATED: start_request will be removed in PyMongo 3.0.
In Python 2.6 and above, or in Python 2.5 with
"from __future__ import with_statement", :meth:`start_request` can be
used as a context manager:
When doing w=0 writes to MongoDB 2.4 or earlier, :meth:`start_request`
was sometimes useful to ensure the current thread always used the same
socket until it called :meth:`end_request`. This made consistent reads
more likely after an unacknowledged write. Requests are no longer
useful in modern MongoDB applications, see
`PYTHON-785 <https://jira.mongodb.org/browse/PYTHON-785>`_.
>>> client = pymongo.MongoClient(auto_start_request=False)
>>> db = client.test
>>> _id = db.test_collection.insert({})
>>> with client.start_request():
... for i in range(100):
... db.test_collection.update({'_id': _id}, {'$set': {'i':i}})
...
... # Definitely read the document after the final update completes
... print db.test_collection.find({'_id': _id})
If a thread or greenlet calls start_request multiple times, an equal
number of calls to :meth:`end_request` is required to end the request.
.. versionchanged:: 2.8
Deprecated.
.. versionchanged:: 2.4
Now counts the number of calls to start_request and doesn't end
@ -1270,17 +1257,17 @@ class MongoClient(common.BaseObject):
return pool.Request(self)
def in_request(self):
"""True if this thread is in a request, meaning it has a socket
reserved for its exclusive use.
"""**DEPRECATED**: True if this thread is in a request, meaning it has
a socket reserved for its exclusive use.
"""
member = self.__member # Don't try to connect if disconnected.
return member and member.in_request()
def end_request(self):
"""Undo :meth:`start_request`. If :meth:`end_request` is called as many
times as :meth:`start_request`, the request is over and this thread's
connection returns to the pool. Extra calls to :meth:`end_request` have
no effect.
"""**DEPRECATED**: Undo :meth:`start_request`. If :meth:`end_request`
is called as many times as :meth:`start_request`, the request is over
and this thread's connection returns to the pool. Extra calls to
:meth:`end_request` have no effect.
Ending a request allows the :class:`~socket.socket` that has
been reserved for this thread by :meth:`start_request` to be returned to

69
pymongo/mongo_replica_set_client.py
View File

@ -503,17 +503,11 @@ class MongoReplicaSetClient(common.BaseObject):
- `socketKeepAlive`: (boolean) Whether to send periodic keep-alive
packets on connected sockets. Defaults to ``False`` (do not send
keep-alive packets).
- `auto_start_request`: If ``True``, each thread that accesses
this :class:`MongoReplicaSetClient` has a socket allocated to it
for the thread's lifetime, for each member of the set. For
:class:`~pymongo.read_preferences.ReadPreference` PRIMARY,
auto_start_request=True ensures consistent reads, even if you read
after an unacknowledged write. For read preferences other than
PRIMARY, there are no consistency guarantees. Default to ``False``.
- `auto_start_request`: Deprecated.
- `use_greenlets`: If ``True``, use a background Greenlet instead of
a background thread to monitor state of replica set. Additionally,
:meth:`start_request()` assigns a greenlet-local, rather than
thread-local, socket. Defaults to ``False``.
a background thread to monitor the state of the replica set.
Additionally, :meth:`start_request` assigns a greenlet-local,
rather than thread-local, socket. Defaults to ``False``.
`use_greenlets` with :class:`MongoReplicaSetClient` requires
`Gevent <http://gevent.org/>`_ to be installed.
@ -1013,8 +1007,7 @@ class MongoReplicaSetClient(common.BaseObject):
@property
def auto_start_request(self):
"""Is auto_start_request enabled?
"""
"""**DEPRECATED** Is auto_start_request enabled?"""
return self.__auto_start_request
def __simple_command(self, sock_info, dbname, spec):
@ -1371,10 +1364,9 @@ class MongoReplicaSetClient(common.BaseObject):
primary, else ``True``.
This method attempts to check the status of the primary with minimal
I/O. The current thread / greenlet retrieves a socket (its request
socket if it's in a request, or a random idle socket if it's not in a
request) from the primary's connection pool and checks whether calling
select_ on it raises an error. If there are currently no idle sockets,
I/O. The current thread / greenlet retrieves a socket from the
primary's connection pool and checks whether calling select_ on it
raises an error. If there are currently no idle sockets,
:meth:`alive` attempts to connect a new socket.
A more certain way to determine primary availability is to ping it::
@ -1733,26 +1725,17 @@ class MongoReplicaSetClient(common.BaseObject):
raise AutoReconnect(str(e))
def start_request(self):
"""Ensure the current thread or greenlet always uses the same socket
until it calls :meth:`end_request`. For
:class:`~pymongo.read_preferences.ReadPreference` PRIMARY,
auto_start_request=True ensures consistent reads, even if you read
after an unacknowledged write. For read preferences other than PRIMARY,
there are no consistency guarantees.
"""DEPRECATED: start_request will be removed in PyMongo 3.0.
In Python 2.6 and above, or in Python 2.5 with
"from __future__ import with_statement", :meth:`start_request` can be
used as a context manager:
When doing w=0 writes to MongoDB 2.4 or earlier, :meth:`start_request`
was sometimes useful to ensure the current thread always used the same
socket until it called :meth:`end_request`. This made consistent reads
more likely after an unacknowledged write. Requests are no longer
useful in modern MongoDB applications, see
`PYTHON-785 <https://jira.mongodb.org/browse/PYTHON-785>`_.
>>> client = pymongo.MongoReplicaSetClient()
>>> db = client.test
>>> _id = db.test_collection.insert({})
>>> with client.start_request():
... for i in range(100):
... db.test_collection.update({'_id': _id}, {'$set': {'i':i}})
...
... # Definitely read the document after the final update completes
... print db.test_collection.find({'_id': _id})
.. versionchanged:: 2.8
Deprecated.
.. versionadded:: 2.2
The :class:`~pymongo.pool.Request` return value.
@ -1771,26 +1754,14 @@ class MongoReplicaSetClient(common.BaseObject):
return pool.Request(self)
def in_request(self):
"""True if :meth:`start_request` has been called, but not
:meth:`end_request`, or if `auto_start_request` is True and
"""**DEPRECATED**: True if :meth:`start_request` has been called, but
not :meth:`end_request`, or if `auto_start_request` is True and
:meth:`end_request` has not been called in this thread or greenlet.
"""
return bool(self.__request_counter.get())
def end_request(self):
"""Undo :meth:`start_request` and allow this thread's connections to
replica set members to return to the pool.
Calling :meth:`end_request` allows the :class:`~socket.socket` that has
been reserved for this thread by :meth:`start_request` to be returned
to the pool. Other threads will then be able to re-use that
:class:`~socket.socket`. If your application uses many threads, or has
long-running threads that infrequently perform MongoDB operations, then
judicious use of this method can lead to performance gains. Care should
be taken, however, to make sure that :meth:`end_request` is not called
in the middle of a sequence of operations in which ordering is
important. This could lead to unexpected results.
"""
"""**DEPRECATED**: Undo :meth:`start_request`."""
rs_state = self.__rs_state
if 0 == self.__request_counter.dec():
for member in rs_state.members:

7
pymongo/replica_set_connection.py
View File

@ -123,11 +123,8 @@ class ReplicaSetConnection(MongoReplicaSetClient):
keep-alive packets).
- `auto_start_request`: If ``True`` (the default), each thread that
accesses this :class:`ReplicaSetConnection` has a socket allocated
to it for the thread's lifetime, for each member of the set. For
:class:`~pymongo.read_preferences.ReadPreference` PRIMARY,
auto_start_request=True ensures consistent reads, even if you read
after an unsafe write. For read preferences other than PRIMARY,
there are no consistency guarantees.
to it for each member of the set until the thread calls
:meth:`end_request` or terminates.
- `use_greenlets`: if ``True``, use a background Greenlet instead of
a background thread to monitor state of replica set. Additionally,
:meth:`start_request()` will ensure that the current greenlet uses