mongodb/mongo-python-driver
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

PYTHON-858 - Clean up high availability docs.

Browse Source
This commit is contained in:
Bernie Hackett 2015-03-11 19:28:54 -07:00
parent f757eb0433
commit 3bcbe0682d
7 changed files with 107 additions and 79 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

141
doc/examples/high_availability.rst
View File

@ -93,16 +93,16 @@ the following connects to the replica set we just created::
>>> MongoClient('mongodb://localhost:27017,localhost:27018/?replicaSet=foo')
MongoClient(['localhost:27017', 'localhost:27018'])
The nodes passed to :meth:`~pymongo.mongo_client.MongoClient` are called
The addresses passed to :meth:`~pymongo.mongo_client.MongoClient` are called
the *seeds*. As long as at least one of the seeds is online, MongoClient
discovers all the members in the replica set, and determines which is the
current primary and which are secondaries.
current primary and which are secondaries or arbiters.
The :class:`~pymongo.mongo_client.MongoClient` constructor is non-blocking:
the constructor returns immediately while the client connects to the replica
set using background threads. Note how, if you create a client and immediately
print its string representation, the client only prints the single host it
knows about. If you wait a moment, it to discovers the whole replica set:
knows about. If you wait a moment, it discovers the whole replica set:
>>> from time import sleep
>>> c = MongoClient(replicaset='foo'); print c; sleep(0.1); print c
@ -125,7 +125,7 @@ example failover to illustrate how everything behaves. First, we'll
connect to the replica set and perform a couple of basic operations::
>>> db = MongoClient("localhost", replicaSet='foo').test
>>> db.test.insert_one({"x": 1})
>>> db.test.insert_one({"x": 1}).inserted_id
ObjectId('...')
>>> db.test.find_one()
{u'x': 1, u'_id': ObjectId('...')}
@ -133,10 +133,8 @@ connect to the replica set and perform a couple of basic operations::
By checking the host and port, we can see that we're connected to
*localhost:27017*, which is the current primary::
>>> db.client.host
'localhost'
>>> db.client.port
27017
>>> db.client.address
('localhost', 27017)
Now let's bring down that node and see what happens when we run our
query again::
@ -155,16 +153,14 @@ might have failed.
On subsequent attempts to run the query we might continue to see this
exception. Eventually, however, the replica set will failover and
elect a new primary (this should take a couple of seconds in
elect a new primary (this should take no more than a couple of seconds in
general). At that point the driver will connect to the new primary and
the operation will succeed::
>>> db.test.find_one()
{u'x': 1, u'_id': ObjectId('...')}
>>> db.client.host
'localhost'
>>> db.client.port
27018
>>> db.client.address
('localhost', 27018)
Bring the former primary back up. It will rejoin the set as a secondary.
Now we can move to the next section: distributing reads to secondaries.
@ -176,59 +172,85 @@ Secondary Reads
By default an instance of MongoClient sends queries to
the primary member of the replica set. To use secondaries for queries
we have to change the :class:`~pymongo.read_preferences.ReadPreference`::
we have to change the read preference::
>>> from pymongo.read_preferences import ReadPreference
>>> client = MongoClient(
... 'localhost:27017',
... replicaSet='foo',
... read_preference=ReadPreference.SECONDARY_PREFERRED)
>>> db = client.test
... readPreference='secondaryPreferred')
>>> client.read_preference
SecondaryPreferred(tag_sets=None)
Now all queries will be sent to the secondary members of the set. If there are
no secondary members the primary will be used as a fallback. If you have
queries you would prefer to never send to the primary you can specify that
using the ``SECONDARY`` read preference.
using the ``secondary`` read preference.
The Read preference can be set when you create a
:class:`~pymongo.mongo_client.MongoClient`, or when you execute a
:meth:`~pymongo.collection.Collection.find`,
:meth:`~pymongo.collection.Collection.find_one`,
or a command::
By default the read preference of a :class:`~pymongo.database.Database` is
inherited from its MongoClient, and the read preference of a
:class:`~pymongo.collection.Collection` is inherited from its Database. To use
a different read preference use the
:meth:`~pymongo.mongo_client.MongoClient.get_database` method, or the
:meth:`~pymongo.database.Database.get_collection` method::
>>> from pymongo import ReadPreference
>>> client.read_preference
SecondaryPreferred(tag_sets=None)
>>> db = client.get_database('test', read_preference=ReadPreference.SECONDARY)
>>> db.read_preference
Secondary(tag_sets=None)
>>> coll = db.get_collection('test', read_preference=ReadPreference.PRIMARY)
>>> coll.read_preference
Primary()
You can also change the read preference of an existing
:class:`~pymongo.collection.Collection` with the
:meth:`~pymongo.collection.Collection.with_options` method::
>>> coll2 = coll.with_options(read_preference=ReadPreference.NEAREST)
>>> coll.read_preference
Primary()
>>> coll2.read_preference
Nearest(tag_sets=None)
Note that since most database commands can only be sent to the primary of a
replica set, the :meth:`~pymongo.database.Database.command` method does not obey
the Database's :attr:`~pymongo.database.Database.read_preference`, but you can
pass an explicit read preference to the method::
>>> db.test.find_one(read_preference=ReadPreference.PRIMARY)
{u'x': 1, u'_id': ObjectId('...')}
>>> for doc in db.test.find(read_preference=ReadPreference.SECONDARY):
... print doc
{u'x': 1, u'_id': ObjectId('...')}
>>> db.command('dbstats', read_preference=ReadPreference.NEAREST)
{...}
Reads are configured using three options: **read_preference**, **tag_sets**,
and **local_threshold_ms**.
Reads are configured using three options: **read preference**, **tag sets**,
and **local threshold**.
**read_preference**:
**Read preference**:
- ``PRIMARY``: Read from the primary. This is the default, and provides the
strongest consistency. If no primary is available, raise
Read preference is configured using one of the classes from
:mod:`~pymongo.read_preferences` (:class:`~pymongo.read_preferences.Primary`,
:class:`~pymongo.read_preferences.PrimaryPreferred`,
:class:`~pymongo.read_preferences.Secondary`,
:class:`~pymongo.read_preferences.SecondaryPreferred`, or
:class:`~pymongo.read_preferences.Nearest`). For convenience, we also provide
:class:`~pymongo.read_preferences.ReadPreference` with the following
attributes:
- ``PRIMARY``: Read from the primary. This is the default read preference,
and provides the strongest consistency. If no primary is available, raise
:class:`~pymongo.errors.AutoReconnect`.
- ``PRIMARY_PREFERRED``: Read from the primary if available, or if there is
none, read from a secondary matching your choice of ``tag_sets`` and
``local_threshold_ms``.
- ``PRIMARY_PREFERRED``: Read from the primary if available, otherwise read
from a secondary.
- ``SECONDARY``: Read from a secondary matching your choice of ``tag_sets`` and
``local_threshold_ms``. If no matching secondary is available,
- ``SECONDARY``: Read from a secondary. If no matching secondary is available,
raise :class:`~pymongo.errors.AutoReconnect`.
- ``SECONDARY_PREFERRED``: Read from a secondary matching your choice of
``tag_sets`` and ``local_threshold_ms`` if available, otherwise
from primary (regardless of the primary's tags and local threshold).
- ``SECONDARY_PREFERRED``: Read from a secondary if available, otherwise
from the primary.
- ``NEAREST``: Read from any member matching your choice of ``tag_sets`` and
``local_threshold_ms``.
- ``NEAREST``: Read from any available member.
**tag_sets**:
**Tag sets**:
Replica-set members can be `tagged
<http://www.mongodb.org/display/DOCS/Data+Center+Awareness>`_ according to any
@ -240,14 +262,13 @@ PyMongo tries each set of tags in turn until it finds a set of
tags with at least one matching member. For example, to prefer reads from the
New York data center, but fall back to the San Francisco data center, tag your
replica set members according to their location and create a
MongoClient like so:
MongoClient like so::
>>> from pymongo.read_preferences import Secondary
>>> rsc = MongoClient(
... 'localhost:27017',
... replicaSet='foo'
... read_preference=Secondary(tag_sets=[{'dc': 'ny'}, {'dc': 'sf'}])
... )
>>> db = client.get_database(
... 'test', read_preference=Secondary([{'dc': 'ny'}, {'dc': 'sf'}]))
>>> db.read_preference
Secondary(tag_sets=[{'dc': 'ny'}, {'dc': 'sf'}])
MongoClient tries to find secondaries in New York, then San Francisco,
and raises :class:`~pymongo.errors.AutoReconnect` if none are available. As an
@ -258,18 +279,24 @@ See :mod:`~pymongo.read_preferences` for more information.
.. _distributes reads to secondaries:
**local_threshold_ms**:
**Local threshold**:
If multiple members match the mode and tag sets, PyMongo reads
If multiple members match the read preference and tag sets, PyMongo reads
from among the nearest members, chosen according to ping time. By default,
only members whose ping times are within 15 milliseconds of the nearest
are used for queries. You can choose to distribute reads among members with
higher latencies by setting ``local_threshold_ms`` to a larger
number. In that case, PyMongo distributes reads among matching
members within ``local_threshold_ms`` of the closest member's
ping time.
higher latencies by setting ``localThresholdMS`` to a larger
number::
.. note:: ``local_threshold_ms`` is ignored when talking to a
>>> client = pymongo.MongoClient(
... replicaSet='repl0',
... readPreference='secondaryPreferred',
... localThresholdMS=35)
In this case, PyMongo distributes reads among matching members within 35
milliseconds of the closest member's ping time.
.. note:: ``localThresholdMS`` is ignored when talking to a
replica set *through* a mongos. The equivalent is the localThreshold_ command
line option.

4
pymongo/collection.py
View File

@ -165,8 +165,8 @@ class Collection(common.BaseObject):
:Parameters:
- `command` - The command itself, as a SON instance.
- `read_preference` (optional) - An subclass of
:class:`~pymongo.read_preferences.ServerMode`.
- `read_preference` (optional) - The read preference for this
command. See :mod:`~pymongo.read_preferences` for options.
- `codec_options` (optional) - An instance of
:class:`~bson.codec_options.CodecOptions`.
- `**kwargs` - any optional keyword arguments accepted by

14
pymongo/common.py
View File

@ -24,7 +24,7 @@ from bson.py3compat import string_type, integer_types
from pymongo.auth import MECHANISMS
from pymongo.errors import ConfigurationError
from pymongo.read_preferences import (read_pref_mode_from_name,
ServerMode)
_ServerMode)
from pymongo.ssl_support import validate_cert_reqs
from pymongo.write_concern import WriteConcern
@ -236,7 +236,7 @@ def validate_timeout_or_zero(option, value):
def validate_read_preference(dummy, value):
"""Validate a read preference.
"""
if not isinstance(value, ServerMode):
if not isinstance(value, _ServerMode):
raise TypeError("%r is not a read preference." % (value,))
return value
@ -441,9 +441,10 @@ class BaseObject(object):
"bson.codec_options.CodecOptions")
self.__codec_options = codec_options
# TODO: Better error reporting for read preference.
if not isinstance(read_preference, ServerMode):
raise TypeError("read_preference is invalid")
if not isinstance(read_preference, _ServerMode):
raise TypeError("%r is not valid for read_preference. See "
"pymongo.read_preferences for valid "
"options." % (read_preference,))
self.__read_preference = read_preference
if not isinstance(write_concern, WriteConcern):
@ -466,8 +467,7 @@ class BaseObject(object):
def read_preference(self):
"""The read preference mode for this instance.
See :class:`~pymongo.read_preferences.ReadPreference` for
available options.
See :mod:`~pymongo.read_preferences` for available options.
.. versionadded:: 2.1
"""

1
pymongo/database.py
View File

@ -395,6 +395,7 @@ class Database(common.BaseObject):
- `allowable_errors`: if `check` is ``True``, error messages
in this list will be ignored by error-checking
- `read_preference`: The read preference for this operation.
See :mod:`~pymongo.read_preferences` for options.
- `codec_options`: A :class:`~bson.codec_options.CodecOptions`
instance.
- `**kwargs` (optional): additional keyword arguments will

18
pymongo/read_preferences.py
View File

@ -62,7 +62,7 @@ def _validate_tag_sets(tag_sets):
return tag_sets
class ServerMode(object):
class _ServerMode(object):
"""Base class for all read preferences.
"""
@ -116,7 +116,7 @@ class ServerMode(object):
self.name, self.__tag_sets)
def __eq__(self, other):
if isinstance(other, ServerMode):
if isinstance(other, _ServerMode):
return (self.mode == other.mode and
self.tag_sets == other.tag_sets)
raise NotImplementedError
@ -125,7 +125,7 @@ class ServerMode(object):
return not self == other
class Primary(ServerMode):
class Primary(_ServerMode):
"""Primary read preference.
* When directly connected to one mongod queries are allowed if the server
@ -143,15 +143,15 @@ class Primary(ServerMode):
return writable_server_selector(server_descriptions)
def __repr__(self):
return "Primary"
return "Primary()"
def __eq__(self, other):
if isinstance(other, ServerMode):
if isinstance(other, _ServerMode):
return other.mode == _PRIMARY
raise NotImplementedError
class PrimaryPreferred(ServerMode):
class PrimaryPreferred(_ServerMode):
"""PrimaryPreferred read preference.
* When directly connected to one mongod queries are allowed to standalone
@ -180,7 +180,7 @@ class PrimaryPreferred(ServerMode):
server_descriptions)
class Secondary(ServerMode):
class Secondary(_ServerMode):
"""Secondary read preference.
* When directly connected to one mongod queries are allowed to standalone
@ -204,7 +204,7 @@ class Secondary(ServerMode):
server_descriptions)
class SecondaryPreferred(ServerMode):
class SecondaryPreferred(_ServerMode):
"""SecondaryPreferred read preference.
* When directly connected to one mongod queries are allowed to standalone
@ -233,7 +233,7 @@ class SecondaryPreferred(ServerMode):
return writable_server_selector(server_descriptions)
class Nearest(ServerMode):
class Nearest(_ServerMode):
"""Nearest read preference.
* When directly connected to one mongod queries are allowed to standalone

6
test/test_read_preferences.py
View File

@ -25,7 +25,7 @@ from pymongo.mongo_client import MongoClient
from pymongo.read_preferences import (ReadPreference, MovingAverage,
Primary, PrimaryPreferred,
Secondary, SecondaryPreferred,
Nearest, ServerMode)
Nearest, _ServerMode)
from pymongo.server_selectors import any_server_selector
from pymongo.server_type import SERVER_TYPE
from pymongo.write_concern import WriteConcern
@ -139,11 +139,11 @@ class TestReadPreferences(TestReadPreferencesBase):
def test_tag_sets_validation(self):
# Can't use tags with PRIMARY
self.assertRaises(ConfigurationError, ServerMode,
self.assertRaises(ConfigurationError, _ServerMode,
0, tag_sets=[{'k': 'v'}])
# ... but empty tag sets are ok with PRIMARY
self.assertRaises(ConfigurationError, ServerMode,
self.assertRaises(ConfigurationError, _ServerMode,
0, tag_sets=[{}])
S = Secondary(tag_sets=[{}])

2
test/test_topology.py
View File

@ -606,7 +606,7 @@ class TestServerSelectionErrors(TopologyTest):
'setName': 'rs',
'hosts': ['a']})
self.assertMessage('No replica set members match selector "Primary"',
self.assertMessage('No replica set members match selector "Primary()"',
t, ReadPreference.PRIMARY)
self.assertMessage('No primary available for writes',