482 lines
17 KiB
ReStructuredText
482 lines
17 KiB
ReStructuredText
.. currentmodule:: motor.motor_asyncio
|
|
|
|
Tutorial: Using Motor With :mod:`asyncio`
|
|
=========================================
|
|
|
|
.. These setups are redundant because I can't figure out how to make doctest
|
|
run a common setup *before* the setup for the two groups. A "testsetup:: *"
|
|
is the obvious answer, but it's run *after* group-specific setup.
|
|
|
|
.. testsetup:: before-inserting-2000-docs
|
|
|
|
import pymongo
|
|
import motor.motor_asyncio
|
|
import asyncio
|
|
|
|
client = motor.motor_asyncio.AsyncIOMotorClient()
|
|
db = client.test_database
|
|
|
|
.. testsetup:: after-inserting-2000-docs
|
|
|
|
import pymongo
|
|
import motor.motor_asyncio
|
|
import asyncio
|
|
|
|
client = motor.motor_asyncio.AsyncIOMotorClient()
|
|
db = client.test_database
|
|
pymongo.MongoClient().test_database.test_collection.insert_many(
|
|
[{"i": i} for i in range(2000)]
|
|
)
|
|
|
|
.. testcleanup:: *
|
|
|
|
import pymongo
|
|
|
|
pymongo.MongoClient().test_database.test_collection.delete_many({})
|
|
|
|
A guide to using MongoDB and asyncio with Motor.
|
|
|
|
.. contents::
|
|
|
|
Tutorial Prerequisites
|
|
----------------------
|
|
You can learn about MongoDB with the `MongoDB Tutorial`_ before you learn Motor.
|
|
|
|
Using Python 3.5 or later, do::
|
|
|
|
$ python3 -m pip install motor
|
|
|
|
This tutorial assumes that a MongoDB instance is running on the
|
|
default host and port. Assuming you have `downloaded and installed
|
|
<https://mongodb.com/docs/manual/installation/>`_ MongoDB, you
|
|
can start it like so:
|
|
|
|
.. code-block:: bash
|
|
|
|
$ mongod
|
|
|
|
.. _pip: http://www.pip-installer.org/en/latest/installing.html
|
|
|
|
.. _MongoDB Tutorial: https://mongodb.com/docs/manual/tutorial/getting-started/
|
|
|
|
Object Hierarchy
|
|
----------------
|
|
Motor, like PyMongo, represents data with a 4-level object hierarchy:
|
|
|
|
* :class:`~motor.motor_asyncio.AsyncIOMotorClient`
|
|
represents a mongod process, or a cluster of them. You explicitly create one
|
|
of these client objects, connect it to a running mongod or mongods, and
|
|
use it for the lifetime of your application.
|
|
* :class:`~motor.motor_asyncio.AsyncIOMotorDatabase`: Each mongod has a set of databases (distinct
|
|
sets of data files on disk). You can get a reference to a database from a
|
|
client.
|
|
* :class:`~motor.motor_asyncio.AsyncIOMotorCollection`: A database has a set of collections, which
|
|
contain documents; you get a reference to a collection from a database.
|
|
* :class:`~motor.motor_asyncio.AsyncIOMotorCursor`: Executing :meth:`~motor.motor_asyncio.AsyncIOMotorCollection.find` on
|
|
an :class:`~motor.motor_asyncio.AsyncIOMotorCollection` gets an :class:`~motor.motor_asyncio.AsyncIOMotorCursor`, which
|
|
represents the set of documents matching a query.
|
|
|
|
Creating a Client
|
|
-----------------
|
|
You typically create a single instance of :class:`~motor.motor_asyncio.AsyncIOMotorClient` at the time your
|
|
application starts up.
|
|
|
|
.. doctest:: before-inserting-2000-docs
|
|
|
|
>>> import motor.motor_asyncio
|
|
>>> client = motor.motor_asyncio.AsyncIOMotorClient()
|
|
|
|
This connects to a ``mongod`` listening on the default host and port. You can
|
|
specify the host and port like:
|
|
|
|
.. doctest:: before-inserting-2000-docs
|
|
|
|
>>> client = motor.motor_asyncio.AsyncIOMotorClient("localhost", 27017)
|
|
|
|
Motor also supports `connection URIs`_:
|
|
|
|
.. doctest:: before-inserting-2000-docs
|
|
|
|
>>> client = motor.motor_asyncio.AsyncIOMotorClient("mongodb://localhost:27017")
|
|
|
|
Connect to a replica set like:
|
|
|
|
>>> client = motor.motor_asyncio.AsyncIOMotorClient('mongodb://host1,host2/?replicaSet=my-replicaset-name')
|
|
|
|
.. _connection URIs: https://mongodb.com/docs/manual/reference/connection-string/
|
|
|
|
Getting a Database
|
|
------------------
|
|
A single instance of MongoDB can support multiple independent
|
|
`databases <https://www.mongodb.com/docs/manual/core/databases-and-collections/>`_.
|
|
From an open client, you can get a reference to a particular database with
|
|
dot-notation or bracket-notation:
|
|
|
|
.. doctest:: before-inserting-2000-docs
|
|
|
|
>>> db = client.test_database
|
|
>>> db = client["test_database"]
|
|
|
|
Creating a reference to a database does no I/O and does not require an
|
|
``await`` expression.
|
|
|
|
Getting a Collection
|
|
--------------------
|
|
A `collection <https://www.mongodb.com/docs/manual/core/databases-and-collections/>`_
|
|
is a group of documents stored in MongoDB, and can be thought of as roughly
|
|
the equivalent of a table in a relational database. Getting a
|
|
collection in Motor works the same as getting a database:
|
|
|
|
.. doctest:: before-inserting-2000-docs
|
|
|
|
>>> collection = db.test_collection
|
|
>>> collection = db["test_collection"]
|
|
|
|
Just like getting a reference to a database, getting a reference to a
|
|
collection does no I/O and doesn't require an ``await`` expression.
|
|
|
|
Inserting a Document
|
|
--------------------
|
|
As in PyMongo, Motor represents MongoDB documents with Python dictionaries. To
|
|
store a document in MongoDB, call :meth:`~AsyncIOMotorCollection.insert_one` in an
|
|
``await`` expression:
|
|
|
|
.. doctest:: before-inserting-2000-docs
|
|
|
|
>>> async def do_insert():
|
|
... document = {"key": "value"}
|
|
... result = await db.test_collection.insert_one(document)
|
|
... print("result %s" % repr(result.inserted_id))
|
|
...
|
|
>>>
|
|
>>> import asyncio
|
|
>>> loop = client.get_io_loop()
|
|
>>> loop.run_until_complete(do_insert())
|
|
result ObjectId('...')
|
|
|
|
.. mongodoc:: insert
|
|
|
|
.. doctest:: before-inserting-2000-docs
|
|
:hide:
|
|
|
|
>>> # Clean up from previous insert
|
|
>>> pymongo.MongoClient().test_database.test_collection.delete_many({})
|
|
DeleteResult({'n': 1, 'ok': 1.0}, acknowledged=True)
|
|
|
|
Insert documents in large batches with :meth:`~AsyncIOMotorCollection.insert_many`:
|
|
|
|
.. doctest:: before-inserting-2000-docs
|
|
|
|
>>> async def do_insert():
|
|
... result = await db.test_collection.insert_many([{"i": i} for i in range(2000)])
|
|
... print("inserted %d docs" % (len(result.inserted_ids),))
|
|
...
|
|
>>> loop = client.get_io_loop()
|
|
>>> loop.run_until_complete(do_insert())
|
|
inserted 2000 docs
|
|
|
|
Getting a Single Document With ``find_one``
|
|
-------------------------------------------
|
|
|
|
Use :meth:`~motor.motor_asyncio.AsyncIOMotorCollection.find_one` to get the first document that
|
|
matches a query. For example, to get a document where the value for key "i" is
|
|
less than 1:
|
|
|
|
.. doctest:: after-inserting-2000-docs
|
|
|
|
>>> async def do_find_one():
|
|
... document = await db.test_collection.find_one({"i": {"$lt": 1}})
|
|
... pprint.pprint(document)
|
|
...
|
|
>>> loop = client.get_io_loop()
|
|
>>> loop.run_until_complete(do_find_one())
|
|
{'_id': ObjectId('...'), 'i': 0}
|
|
|
|
The result is a dictionary matching the one that we inserted previously.
|
|
|
|
.. note:: The returned document contains an ``"_id"``, which was
|
|
automatically added on insert.
|
|
|
|
.. mongodoc:: find
|
|
|
|
Querying for More Than One Document
|
|
-----------------------------------
|
|
Use :meth:`~motor.motor_asyncio.AsyncIOMotorCollection.find` to query for a set of documents.
|
|
:meth:`~motor.motor_asyncio.AsyncIOMotorCollection.find` does no I/O and does not require an ``await``
|
|
expression. It merely creates an :class:`~motor.motor_asyncio.AsyncIOMotorCursor` instance. The query is
|
|
actually executed on the server when you call :meth:`~motor.motor_asyncio.AsyncIOMotorCursor.to_list`
|
|
or execute an ``async for`` loop.
|
|
|
|
To find all documents with "i" less than 5:
|
|
|
|
.. doctest:: after-inserting-2000-docs
|
|
|
|
>>> async def do_find():
|
|
... cursor = db.test_collection.find({"i": {"$lt": 5}}).sort("i")
|
|
... for document in await cursor.to_list(length=100):
|
|
... pprint.pprint(document)
|
|
...
|
|
>>> loop = client.get_io_loop()
|
|
>>> loop.run_until_complete(do_find())
|
|
{'_id': ObjectId('...'), 'i': 0}
|
|
{'_id': ObjectId('...'), 'i': 1}
|
|
{'_id': ObjectId('...'), 'i': 2}
|
|
{'_id': ObjectId('...'), 'i': 3}
|
|
{'_id': ObjectId('...'), 'i': 4}
|
|
|
|
A ``length`` argument is required when you call ``to_list`` to prevent Motor
|
|
from buffering an unlimited number of documents.
|
|
|
|
``async for``
|
|
~~~~~~~~~~~~~
|
|
|
|
You can handle one document at a time in an ``async for`` loop:
|
|
|
|
.. doctest:: after-inserting-2000-docs
|
|
|
|
>>> async def do_find():
|
|
... c = db.test_collection
|
|
... async for document in c.find({"i": {"$lt": 2}}):
|
|
... pprint.pprint(document)
|
|
...
|
|
>>> loop = client.get_io_loop()
|
|
>>> loop.run_until_complete(do_find())
|
|
{'_id': ObjectId('...'), 'i': 0}
|
|
{'_id': ObjectId('...'), 'i': 1}
|
|
|
|
You can apply a sort, limit, or skip to a query before you begin iterating:
|
|
|
|
.. doctest:: after-inserting-2000-docs
|
|
|
|
>>> async def do_find():
|
|
... cursor = db.test_collection.find({"i": {"$lt": 4}})
|
|
... # Modify the query before iterating
|
|
... cursor.sort("i", -1).skip(1).limit(2)
|
|
... async for document in cursor:
|
|
... pprint.pprint(document)
|
|
...
|
|
>>> loop = client.get_io_loop()
|
|
>>> loop.run_until_complete(do_find())
|
|
{'_id': ObjectId('...'), 'i': 2}
|
|
{'_id': ObjectId('...'), 'i': 1}
|
|
|
|
The cursor does not actually retrieve each document from the server
|
|
individually; it gets documents efficiently in `large batches`_.
|
|
|
|
.. _`large batches`: https://mongodb.com/docs/manual/tutorial/iterate-a-cursor/#cursor-batches
|
|
|
|
Counting Documents
|
|
------------------
|
|
Use :meth:`~motor.motor_asyncio.AsyncIOMotorCollection.count_documents` to
|
|
determine the number of documents in a collection, or the number of documents
|
|
that match a query:
|
|
|
|
.. doctest:: after-inserting-2000-docs
|
|
|
|
>>> async def do_count():
|
|
... n = await db.test_collection.count_documents({})
|
|
... print("%s documents in collection" % n)
|
|
... n = await db.test_collection.count_documents({"i": {"$gt": 1000}})
|
|
... print("%s documents where i > 1000" % n)
|
|
...
|
|
>>> loop = client.get_io_loop()
|
|
>>> loop.run_until_complete(do_count())
|
|
2000 documents in collection
|
|
999 documents where i > 1000
|
|
|
|
Updating Documents
|
|
------------------
|
|
|
|
:meth:`~motor.motor_asyncio.AsyncIOMotorCollection.replace_one` changes a document. It requires two
|
|
parameters: a *query* that specifies which document to replace, and a
|
|
replacement document. The query follows the same syntax as for :meth:`find` or
|
|
:meth:`find_one`. To replace a document:
|
|
|
|
.. doctest:: after-inserting-2000-docs
|
|
|
|
>>> async def do_replace():
|
|
... coll = db.test_collection
|
|
... old_document = await coll.find_one({"i": 50})
|
|
... print("found document: %s" % pprint.pformat(old_document))
|
|
... _id = old_document["_id"]
|
|
... result = await coll.replace_one({"_id": _id}, {"key": "value"})
|
|
... print("replaced %s document" % result.modified_count)
|
|
... new_document = await coll.find_one({"_id": _id})
|
|
... print("document is now %s" % pprint.pformat(new_document))
|
|
...
|
|
>>> loop = client.get_io_loop()
|
|
>>> loop.run_until_complete(do_replace())
|
|
found document: {'_id': ObjectId('...'), 'i': 50}
|
|
replaced 1 document
|
|
document is now {'_id': ObjectId('...'), 'key': 'value'}
|
|
|
|
You can see that :meth:`replace_one` replaced everything in the old document
|
|
except its ``_id`` with the new document.
|
|
|
|
Use :meth:`~motor.motor_asyncio.AsyncIOMotorCollection.update_one` with MongoDB's modifier operators to
|
|
update part of a document and leave the
|
|
rest intact. We'll find the document whose "i" is 51 and use the ``$set``
|
|
operator to set "key" to "value":
|
|
|
|
.. doctest:: after-inserting-2000-docs
|
|
|
|
>>> async def do_update():
|
|
... coll = db.test_collection
|
|
... result = await coll.update_one({"i": 51}, {"$set": {"key": "value"}})
|
|
... print("updated %s document" % result.modified_count)
|
|
... new_document = await coll.find_one({"i": 51})
|
|
... print("document is now %s" % pprint.pformat(new_document))
|
|
...
|
|
>>> loop = client.get_io_loop()
|
|
>>> loop.run_until_complete(do_update())
|
|
updated 1 document
|
|
document is now {'_id': ObjectId('...'), 'i': 51, 'key': 'value'}
|
|
|
|
"key" is set to "value" and "i" is still 51.
|
|
|
|
:meth:`update_one` only affects the first document it finds, you can
|
|
update all of them with :meth:`update_many`::
|
|
|
|
await coll.update_many({'i': {'$gt': 100}},
|
|
{'$set': {'key': 'value'}})
|
|
|
|
.. mongodoc:: update
|
|
|
|
Deleting Documents
|
|
------------------
|
|
|
|
:meth:`~motor.motor_asyncio.AsyncIOMotorCollection.delete_one` takes a query with the same syntax as
|
|
:meth:`~motor.motor_asyncio.AsyncIOMotorCollection.find`.
|
|
:meth:`delete_one` immediately removes the first returned matching document.
|
|
|
|
.. doctest:: after-inserting-2000-docs
|
|
|
|
>>> async def do_delete_one():
|
|
... coll = db.test_collection
|
|
... n = await coll.count_documents({})
|
|
... print("%s documents before calling delete_one()" % n)
|
|
... result = await db.test_collection.delete_one({"i": {"$gte": 1000}})
|
|
... print("%s documents after" % (await coll.count_documents({})))
|
|
...
|
|
>>> loop = client.get_io_loop()
|
|
>>> loop.run_until_complete(do_delete_one())
|
|
2000 documents before calling delete_one()
|
|
1999 documents after
|
|
|
|
:meth:`~motor.motor_asyncio.AsyncIOMotorCollection.delete_many` takes a query with the same syntax as
|
|
:meth:`~motor.motor_asyncio.AsyncIOMotorCollection.find`.
|
|
:meth:`delete_many` immediately removes all matching documents.
|
|
|
|
.. doctest:: after-inserting-2000-docs
|
|
|
|
>>> async def do_delete_many():
|
|
... coll = db.test_collection
|
|
... n = await coll.count_documents({})
|
|
... print("%s documents before calling delete_many()" % n)
|
|
... result = await db.test_collection.delete_many({"i": {"$gte": 1000}})
|
|
... print("%s documents after" % (await coll.count_documents({})))
|
|
...
|
|
>>> loop = client.get_io_loop()
|
|
>>> loop.run_until_complete(do_delete_many())
|
|
1999 documents before calling delete_many()
|
|
1000 documents after
|
|
|
|
.. mongodoc:: remove
|
|
|
|
Commands
|
|
--------
|
|
All operations on MongoDB are implemented internally as commands. Run them using
|
|
the :meth:`~motor.motor_asyncio.AsyncIOMotorDatabase.command` method on
|
|
:class:`~motor.motor_asyncio.AsyncIOMotorDatabase`::
|
|
|
|
.. doctest:: after-inserting-2000-docs
|
|
|
|
>>> from bson import SON
|
|
>>> async def use_distinct_command():
|
|
... response = await db.command(SON([("distinct", "test_collection"), ("key", "i")]))
|
|
...
|
|
>>> loop = client.get_io_loop()
|
|
>>> loop.run_until_complete(use_distinct_command())
|
|
|
|
Since the order of command parameters matters, don't use a Python dict to pass
|
|
the command's parameters. Instead, make a habit of using :class:`bson.SON`,
|
|
from the ``bson`` module included with PyMongo.
|
|
|
|
Many commands have special helper methods, such as
|
|
:meth:`~motor.motor_asyncio.AsyncIOMotorDatabase.create_collection` or
|
|
:meth:`~motor.motor_asyncio.AsyncIOMotorCollection.aggregate`, but these are just conveniences atop
|
|
the basic :meth:`command` method.
|
|
|
|
.. mongodoc:: commands
|
|
|
|
.. _example-web-application-aiohttp:
|
|
|
|
A Web Application With `aiohttp`_
|
|
---------------------------------
|
|
|
|
Let us create a web application using `aiohttp`_, a popular HTTP package for
|
|
asyncio. Install it with::
|
|
|
|
python3 -m pip install aiohttp
|
|
|
|
We are going to make a trivial web site with two pages served from MongoDB.
|
|
To begin:
|
|
|
|
.. literalinclude:: examples/aiohttp_example.py
|
|
:language: python3
|
|
:start-after: setup-start
|
|
:end-before: setup-end
|
|
|
|
The ``AsyncIOMotorClient`` constructor does not actually connect to MongoDB.
|
|
The client connects on demand, when you attempt the first operation.
|
|
We create it and assign the "test" database's handle to ``db``.
|
|
|
|
The ``setup_db`` coroutine drops the "pages" collection (plainly, this code is
|
|
for demonstration purposes), then inserts two documents. Each document's page
|
|
name is its unique id, and the "body" field is a simple HTML page. Finally,
|
|
``setup_db`` returns the database handle.
|
|
|
|
We'll use the ``setup_db`` coroutine soon. First, we need a request handler
|
|
that serves pages from the data we stored in MongoDB.
|
|
|
|
.. literalinclude:: examples/aiohttp_example.py
|
|
:language: python3
|
|
:start-after: handler-start
|
|
:end-before: handler-end
|
|
|
|
We start the server by running ``setup_db`` and passing the database handle
|
|
to an :class:`aiohttp.web.Application`:
|
|
|
|
.. literalinclude:: examples/aiohttp_example.py
|
|
:language: python3
|
|
:start-after: main-start
|
|
:end-before: main-end
|
|
|
|
Note that it is a common mistake to create a new client object for every
|
|
request; this comes at a dire performance cost. Create the client
|
|
when your application starts and reuse that one client for the lifetime
|
|
of the process. You can maintain the client by storing a database handle
|
|
from the client on your application object, as shown in this example.
|
|
|
|
Visit ``localhost:8080/pages/page-one`` and the server responds "Hello!".
|
|
At ``localhost:8080/pages/page-two`` it responds "Goodbye." At other URLs it
|
|
returns a 404.
|
|
|
|
The complete code is in the Motor repository in ``examples/aiohttp_example.py``.
|
|
|
|
.. _aiohttp: https://aiohttp.readthedocs.io/
|
|
|
|
See also the :doc:`examples/aiohttp_gridfs_example`.
|
|
|
|
Further Reading
|
|
---------------
|
|
The handful of classes and methods introduced here are sufficient for daily
|
|
tasks. The API documentation for :class:`~motor.motor_asyncio.AsyncIOMotorClient`, :class:`~motor.motor_asyncio.AsyncIOMotorDatabase`,
|
|
:class:`~motor.motor_asyncio.AsyncIOMotorCollection`, and :class:`~motor.motor_asyncio.AsyncIOMotorCursor` provides a
|
|
reference to Motor's complete feature set.
|
|
|
|
Learning to use the MongoDB driver is just the beginning, of course. For
|
|
in-depth instruction in MongoDB itself, see `The MongoDB Manual`_.
|
|
|
|
.. _The MongoDB Manual: https://mongodb.com/docs/manual/
|