diff --git a/doc/index.rst b/doc/index.rst index ada6a87ee..d8d519471 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -22,6 +22,9 @@ everything you need to know to use **PyMongo**. :doc:`faq` Some questions that come up often. +:doc:`migrate-to-pymongo3` + A PyMongo 2.x to 3.x migration guide. + :doc:`python3` Frequently asked questions about python 3 support. @@ -91,4 +94,5 @@ Indices and tables contributors changelog python3 + migrate-to-pymongo3 developer/index diff --git a/doc/migrate-to-pymongo3.rst b/doc/migrate-to-pymongo3.rst new file mode 100644 index 000000000..8a0e1e3a5 --- /dev/null +++ b/doc/migrate-to-pymongo3.rst @@ -0,0 +1,542 @@ +PyMongo 3 Migration Guide +========================= + +.. contents:: + +.. testsetup:: + + from pymongo import MongoClient, ReadPreference + client = MongoClient() + collection = client.my_database.my_collection + +PyMongo 3 is a partial rewrite bringing a large number of improvements. It +also brings a number of backward breaking changes. This guide provides a +roadmap for migrating an existing application from PyMongo 2.x to 3.x or +writing libraries that will work with both PyMongo 2.x and 3.x. + +PyMongo 2.9 +----------- + +The first step in any successful migration involves upgrading to, or +requiring, at least PyMongo 2.9. If your project has a +requirements.txt file, add the line "pymongo >= 2.9, < 3.0" until you have +completely migrated to PyMongo 3. Most of the key new +methods and options from PyMongo 3.0 are backported in PyMongo 2.9 making +migration much easier. + +Enable Deprecation Warnings +--------------------------- + +Starting with PyMongo 2.9, :exc:`DeprecationWarning` is raised by most methods +removed in PyMongo 3.0. Make sure you enable runtime warnings to see +where deprecated functions and methods are being used in your application:: + + python -Wd + +Warnings can also be changed to errors:: + + python -Wd -Werror + +CRUD API +-------- + +Changes to find() and find_one() +................................ + +"spec" renamed "filter" +~~~~~~~~~~~~~~~~~~~~~~~ + +The `spec` option has been renamed to `filter`. Code like this:: + + >>> cursor = collection.find(spec={"a": 1}) + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + >>> cursor = collection.find(filter={"a": 1}) + +or this with any version of PyMongo: + +.. doctest:: + + >>> cursor = collection.find({"a": 1}) + +"fields" renamed "projection" +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The `fields` option has been renamed to `projection`. Code like this:: + + >>> cursor = collection.find({"a": 1}, fields={"_id": False}) + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + >>> cursor = collection.find({"a": 1}, projection={"_id": False}) + +or this with any version of PyMongo: + +.. doctest:: + + >>> cursor = collection.find({"a": 1}, {"_id": False}) + +"partial" renamed "allow_partial_results" +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The `partial` option has been renamed to `allow_partial_results`. Code like +this:: + + >>> cursor = collection.find({"a": 1}, partial=True) + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + >>> cursor = collection.find({"a": 1}, allow_partial_results=True) + +"timeout" replaced by "no_cursor_timeout" +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The `timeout` option has been replaced by `no_cursor_timeout`. Code like this:: + + >>> cursor = collection.find({"a": 1}, timeout=False) + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + >>> cursor = collection.find({"a": 1}, no_cursor_timeout=True) + +"snapshot" and "max_scan" replaced by "modifiers" +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The `snapshot` and `max_scan` options have been removed. They can now be set, +along with other $ query modifiers, through the `modifiers` option. Code like +this:: + + >>> cursor = collection.find({"a": 1}, snapshot=True) + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + >>> cursor = collection.find({"a": 1}, modifiers={"$snapshot": True}) + +or with any version of PyMongo: + +.. doctest:: + + >>> cursor = collection.find({"$query": {"a": 1}, "$snapshot": True}) + +"network_timeout" is removed +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The `network_timeout` option has been removed. This option was always the +wrong solution for timing out long running queries and should never be used +in production. Starting with **MongoDB 2.6** you can use the $maxTimeMS query +modifier. Code like this:: + + # Set a 5 second select() timeout. + >>> cursor = collection.find({"a": 1}, network_timeout=5) + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + # Set a 5 second (5000 millisecond) server side query timeout. + >>> cursor = collection.find({"a": 1}, modifiers={"$maxTimeMS": 5000}) + +or with any version of PyMongo: + +.. doctest:: + + >>> cursor = collection.find({"$query": {"a": 1}, "$maxTimeMS": 5000}) + +.. seealso:: `$maxTimeMS + `_ + +Tailable cursors +~~~~~~~~~~~~~~~~ + +The `tailable` and `await_data` options have been replaced by `cursor_type`. +Code like this:: + + >>> cursor = collection.find({"a": 1}, tailable=True) + >>> cursor = collection.find({"a": 1}, tailable=True, await_data=True) + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + >>> from pymongo import CursorType + >>> cursor = collection.find({"a": 1}, cursor_type=CursorType.TAILABLE) + >>> cursor = collection.find({"a": 1}, cursor_type=CursorType.TAILABLE_AWAIT) + +Other removed options +~~~~~~~~~~~~~~~~~~~~~ + +The `slave_okay`, `read_preference`, `tag_sets`, +and `secondary_acceptable_latency_ms` options have been removed. See the `Read +Preferences`_ section for solutions. + +The aggregate method always returns a cursor +............................................ + +PyMongo 2.6 added an option to return an iterable cursor from +:meth:`~pymongo.collection.Collection.aggregate`. In PyMongo 3 +:meth:`~pymongo.collection.Collection.aggregate` always returns a cursor. Use +the `cursor` option for consistent behavior with PyMongo 2.9 and later: + +.. doctest:: + + >>> for result in collection.aggregate([], cursor={}): + ... pass + +Read Preferences +---------------- + +The "slave_okay" option is removed +.................................. + +The `slave_okay` option is removed from PyMongo's API. The +secondaryPreferred read preference provides the same behavior. +Code like this:: + + >>> client = MongoClient(slave_okay=True) + +can be changed to this with PyMongo 2.9 or newer: + +.. doctest:: + + >>> client = MongoClient(readPreference="secondaryPreferred") + +The "read_preference" attribute is immutable +............................................ + +Code like this:: + + >>> from pymongo import ReadPreference + >>> db = client.my_database + >>> db.read_preference = ReadPreference.SECONDARY + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + >>> db = client.get_database("my_database", + ... read_preference=ReadPreference.SECONDARY) + +Code like this:: + + >>> cursor = collection.find({"a": 1}, + ... read_preference=ReadPreference.SECONDARY) + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + >>> coll2 = collection.with_options(read_preference=ReadPreference.SECONDARY) + >>> cursor = coll2.find({"a": 1}) + +.. seealso:: :meth:`~pymongo.database.Database.get_collection` + +The "tag_sets" attribute is removed +................................... + +Code like this:: + + >>> db = client.my_database + >>> db.read_preference = ReadPreference.SECONDARY + >>> db.tag_sets = [{"dc": "ny"}] + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + >>> from pymongo.read_preferences import Secondary + >>> db = client.get_database("my_database", + ... read_preference=Secondary([{"dc": "ny"}])) + +Code like this:: + + >>> cursor = collection.find( + ... {"a": 1}, + ... read_preference=ReadPreference.SECONDARY, + ... tag_sets=[{"dc": "ny"}]) + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + >>> from pymongo.read_preferences import Secondary + >>> coll2 = collection.with_options( + ... read_preference=Secondary([{"dc": "ny"}])) + >>> cursor = coll2.find({"a": 1}) + +.. seealso:: :meth:`~pymongo.database.Database.get_collection` + +The "secondary_acceptable_latency_ms" option and attribute are removed +...................................................................... + +PyMongo 2.x supports `secondary_acceptable_latency_ms` as an option to methods +throughout the driver, but mongos only supports a global latency option. +PyMongo 3.x has changed to match the behavior of mongos, allowing migration +from a single server, to a replica set, to a sharded cluster without a +surprising change in server selection behavior. A new option, +`localThresholdMS`, is available through MongoClient and should be used in +place of `secondaryAcceptableLatencyMS`. Code like this:: + + >>> client = MongoClient(readPreference="nearest", + ... secondaryAcceptableLatencyMS=100) + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + >>> client = MongoClient(readPreference="nearest", + ... localThresholdMS=100) + +Write Concern +------------- + +The "safe" option is removed +............................ + +In PyMongo 3 the `safe` option is removed from the entire API. +:class:`~pymongo.mongo_client.MongoClient` has always defaulted to acknowledged +write operations and continues to do so in PyMongo 3. + +The "write_concern" attribute is immutable +.......................................... + +The `write_concern` attribute is immutable in PyMongo 3. Code like this:: + + >>> client = MongoClient() + >>> client.write_concern = {"w": "majority"} + +can be changed to this with any version of PyMongo: + +.. doctest:: + + >>> client = MongoClient(w="majority") + +Code like this:: + + >>> db = client.my_database + >>> db.write_concern = {"w": "majority"} + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + >>> from pymongo import WriteConcern + >>> db = client.get_database("my_database", + ... write_concern=WriteConcern(w="majority")) + +The new CRUD API write methods do not accept write concern options. Code like +this:: + + >>> oid = collection.insert({"a": 2}, w="majority") + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + >>> from pymongo import WriteConcern + >>> coll2 = collection.with_options( + ... write_concern=WriteConcern(w="majority")) + >>> oid = coll2.insert({"a": 2}) + +.. seealso:: :meth:`~pymongo.database.Database.get_collection` + +Codec Options +------------- + +The "document_class" attribute is removed +......................................... + +Code like this:: + + >>> from bson.son import SON + >>> client = MongoClient() + >>> client.document_class = SON + +can be replaced by this in any version of PyMongo: + +.. doctest:: + + >>> from bson.son import SON + >>> client = MongoClient(document_class=SON) + +or to change the `document_class` for a :class:`~pymongo.database.Database` +with PyMongo 2.9 or later: + +.. doctest:: + + >>> from bson.codec_options import CodecOptions + >>> from bson.son import SON + >>> db = client.get_database("my_database", CodecOptions(SON)) + +.. seealso:: :meth:`~pymongo.database.Database.get_collection` and + :meth:`~pymongo.collection.Collection.with_options` + +The "uuid_subtype" option and attribute are removed +................................................... + +Code like this:: + + >>> from bson.binary import JAVA_LEGACY + >>> db = client.my_database + >>> db.uuid_subtype = JAVA_LEGACY + +can be replaced by this with PyMongo 2.9 or later: + +.. doctest:: + + >>> from bson.binary import JAVA_LEGACY + >>> from bson.codec_options import CodecOptions + >>> db = client.get_database("my_database", + ... CodecOptions(uuid_representation=JAVA_LEGACY)) + +.. seealso:: :meth:`~pymongo.database.Database.get_collection` and + :meth:`~pymongo.collection.Collection.with_options` + +MongoClient +----------- + +MongoClient connects asynchronously +................................... + +In PyMongo 3, the :class:`~pymongo.mongo_client.MongoClient` constructor no +longer blocks while connecting to the server or servers, and it no longer +raises :exc:`~pymongo.errors.ConnectionFailure` if they are unavailable, nor +:exc:`~pymongo.errors.ConfigurationError` if the user’s credentials are wrong. +Instead, the constructor returns immediately and launches the connection +process on background threads. The `connect` option is added to control whether +these threads are started immediately, or when the client is first used. + +For consistent behavior in PyMongo 2.x and PyMongo 3.x, code like this:: + + >>> from pymongo.errors import ConnectionFailure + >>> try: + ... client = MongoClient() + ... except ConnectionFailure: + ... print("Server not available") + >>> + +can be changed to this with PyMongo 2.9 or later: + +.. doctest:: + + >>> from pymongo.errors import ConnectionFailure + >>> client = MongoClient(connect=False) + >>> try: + ... result = client.admin.command("ismaster") + ... except ConnectionFailure: + ... print("Server not available") + >>> + +Any operation can be used to determine if the server is available. We choose +the "ismaster" command here because it is cheap and does not require auth, so +it is a simple way to check whether the server is available. + +The max_pool_size parameter is removed +...................................... + +PyMongo 3 replaced the max_pool_size parameter with support for the MongoDB URI +`maxPoolSize` option. Code like this:: + + >>> client = MongoClient(max_pool_size=10) + +can be replaced by this with PyMongo 2.9 or later: + +.. doctest:: + + >>> client = MongoClient(maxPoolSize=10) + >>> client = MongoClient("mongodb://localhost:27017/?maxPoolSize=10") + +The "disconnect" method is removed +.................................. + +Code like this:: + + >>> client.disconnect() + +can be replaced by this with PyMongo 2.9 or later: + +.. doctest:: + + >>> client.close() + +The host and port attributes are removed +........................................ + +Code like this:: + + >>> host = client.host + >>> port = client.port + +can be replaced by this with PyMongo 2.9 or later: + +.. doctest:: + + >>> address = client.address + >>> host, port = address or (None, None) + +BSON +---- + +"as_class", "tz_aware", and "uuid_subtype" are removed +...................................................... + +The `as_class`, `tz_aware`, and `uuid_subtype` parameters have been +removed from the functions provided in :mod:`bson`. Code like this:: + + >>> from bson import BSON + >>> from bson.son import SON + >>> encoded = BSON.encode({"a": 1}, as_class=SON) + +can be replaced by this in PyMongo 2.9 or later: + +.. doctest:: + + >>> from bson import BSON + >>> from bson.codec_options import CodecOptions + >>> from bson.son import SON + >>> encoded = BSON.encode({"a": 1}, codec_options=CodecOptions(SON)) + +Advice for features removed in PyMongo 3 +---------------------------------------- + +MasterSlaveConnection is removed +................................ + +Master slave deployments are deprecated in MongoDB. Starting with MongoDB 3.0 +a replica set can have up to 50 members and that limit is likely to be +removed in later releases. We recommend migrating to replica sets instead. + +Requests are removed +.................... + +The client methods `start_request`, `in_request`, and `end_request` are +removed. Requests were designed to make read-your-writes consistency more +likely with the w=0 write concern. Additionally, a thread in a request used the +same member for all secondary reads in a replica set. To ensure +read-your-writes consistency in PyMongo 3.0, do not override the default write +concern with w=0, and do not override the default read preference of PRIMARY. + +The "compile_re" option is removed +.................................. + +In PyMongo 3 regular expressions are never compiled to Python match objects. + +The "use_greenlets" option is removed +..................................... + +The `use_greenlets` option was meant to allow use of PyMongo with Gevent +without the use of gevent.monkey.patch_threads(). This option caused a lot +of confusion and made it difficult to support alternative asyncio libraries +like Eventlet. Users of Gevent should use gevent.monkey.patch_all() instead. + +.. seealso:: :doc:`examples/gevent`