motor/doc/tutorial-asyncio.rst

.. currentmodule:: motor.motor_asyncio

Tutorial: Using Motor With :mod:`asyncio`
=========================================

.. warning:: Motor will be deprecated on May 14th, 2026, one year after the production release of the PyMongo Async driver. Critical bug fixes will be made until May 14th, 2027.
  We strongly recommend that Motor users migrate to the PyMongo Async driver while Motor is still supported.
  To learn more, see `the migration guide <https://www.mongodb.com/docs/languages/python/pymongo-driver/current/reference/migration/>`_.


.. These setups are redundant because I can't figure out how to make doctest
  run a common setup *before* the setup for the two groups. A "testsetup:: *"
  is the obvious answer, but it's run *after* group-specific setup.

.. testsetup:: before-inserting-2000-docs

  import pymongo
  import motor.motor_asyncio
  import asyncio

  client = motor.motor_asyncio.AsyncIOMotorClient()
  db = client.test_database

.. testsetup:: after-inserting-2000-docs

  import pymongo
  import motor.motor_asyncio
  import asyncio

  client = motor.motor_asyncio.AsyncIOMotorClient()
  db = client.test_database
  pymongo.MongoClient().test_database.test_collection.insert_many(
      [{"i": i} for i in range(2000)]
  )

.. testcleanup:: *

  import pymongo

  pymongo.MongoClient().test_database.test_collection.delete_many({})

A guide to using MongoDB and asyncio with Motor.

.. contents::

Tutorial Prerequisites
----------------------
You can learn about MongoDB with the `MongoDB Tutorial`_ before you learn Motor.

Using Python 3.5 or later, do::

  $ python3 -m pip install motor

This tutorial assumes that a MongoDB instance is running on the
default host and port. Assuming you have `downloaded and installed
<https://mongodb.com/docs/manual/installation/>`_ MongoDB, you
can start it like so:

.. code-block:: bash

  $ mongod

.. _pip: http://www.pip-installer.org/en/latest/installing.html

.. _MongoDB Tutorial: https://mongodb.com/docs/manual/tutorial/getting-started/

Object Hierarchy
----------------
Motor, like PyMongo, represents data with a 4-level object hierarchy:

* :class:`~motor.motor_asyncio.AsyncIOMotorClient`
  represents a mongod process, or a cluster of them. You explicitly create one
  of these client objects, connect it to a running mongod or mongods, and
  use it for the lifetime of your application.
* :class:`~motor.motor_asyncio.AsyncIOMotorDatabase`: Each mongod has a set of databases (distinct
  sets of data files on disk). You can get a reference to a database from a
  client.
* :class:`~motor.motor_asyncio.AsyncIOMotorCollection`: A database has a set of collections, which
  contain documents; you get a reference to a collection from a database.
* :class:`~motor.motor_asyncio.AsyncIOMotorCursor`: Executing :meth:`~motor.motor_asyncio.AsyncIOMotorCollection.find` on
  an :class:`~motor.motor_asyncio.AsyncIOMotorCollection` gets an :class:`~motor.motor_asyncio.AsyncIOMotorCursor`, which
  represents the set of documents matching a query.

Creating a Client
-----------------
Creating a client is what establishes a connection to MongoDB and tells your
app what deployment (i.e. cluster) to connect to. You typically create a single
instance of :class:`~motor.motor_asyncio.AsyncIOMotorClient` at the time your
application starts up.

.. doctest:: before-inserting-2000-docs

  >>> import motor.motor_asyncio
  >>> client = motor.motor_asyncio.AsyncIOMotorClient()

This connects to a ``mongod`` listening on the default host and port. You can
specify the host and port like:

.. doctest:: before-inserting-2000-docs

  >>> client = motor.motor_asyncio.AsyncIOMotorClient("localhost", 27017)

Motor also supports `connection URIs`_:

.. doctest:: before-inserting-2000-docs

  >>> client = motor.motor_asyncio.AsyncIOMotorClient("mongodb://localhost:27017")

Connect to a replica set like:

  >>> client = motor.motor_asyncio.AsyncIOMotorClient('mongodb://host1,host2/?replicaSet=my-replicaset-name')

.. _connection URIs: https://mongodb.com/docs/manual/reference/connection-string/

Getting a Database
------------------
A single instance of MongoDB can support multiple independent
`databases <https://www.mongodb.com/docs/manual/core/databases-and-collections/>`_.
From an open client, you can get a reference to a particular database with
dot-notation or bracket-notation:

.. doctest:: before-inserting-2000-docs

  >>> db = client.test_database
  >>> db = client["test_database"]

Creating a reference to a database does no I/O and does not require an
``await`` expression.

Getting a Collection
--------------------
A `collection <https://www.mongodb.com/docs/manual/core/databases-and-collections/>`_
is a group of documents stored in MongoDB, and can be thought of as roughly
the equivalent of a table in a relational database. Getting a
collection in Motor works the same as getting a database:

.. doctest:: before-inserting-2000-docs

  >>> collection = db.test_collection
  >>> collection = db["test_collection"]

Just like getting a reference to a database, getting a reference to a
collection does no I/O and doesn't require an ``await`` expression.

Inserting a Document
--------------------
As in PyMongo, Motor represents MongoDB documents with Python dictionaries. To
store a document in MongoDB, call :meth:`~AsyncIOMotorCollection.insert_one` in an
``await`` expression:

.. doctest:: before-inserting-2000-docs

  >>> async def do_insert():
  ...     document = {"key": "value"}
  ...     result = await db.test_collection.insert_one(document)
  ...     print("result %s" % repr(result.inserted_id))
  ...
  >>>
  >>> import asyncio
  >>> loop = client.get_io_loop()
  >>> loop.run_until_complete(do_insert())
  result ObjectId('...')

.. mongodoc:: insert

.. doctest:: before-inserting-2000-docs
  :hide:

  >>> # Clean up from previous insert
  >>> pymongo.MongoClient().test_database.test_collection.delete_many({})
  DeleteResult({'n': 1, 'ok': 1.0}, acknowledged=True)

Insert documents in large batches with :meth:`~AsyncIOMotorCollection.insert_many`:

.. doctest:: before-inserting-2000-docs

  >>> async def do_insert():
  ...     result = await db.test_collection.insert_many([{"i": i} for i in range(2000)])
  ...     print("inserted %d docs" % (len(result.inserted_ids),))
  ...
  >>> loop = client.get_io_loop()
  >>> loop.run_until_complete(do_insert())
  inserted 2000 docs

Getting a Single Document With ``find_one``
-------------------------------------------

Use :meth:`~motor.motor_asyncio.AsyncIOMotorCollection.find_one` to get the first document that
matches a query. For example, to get a document where the value for key "i" is
less than 1:

.. doctest:: after-inserting-2000-docs

  >>> async def do_find_one():
  ...     document = await db.test_collection.find_one({"i": {"$lt": 1}})
  ...     pprint.pprint(document)
  ...
  >>> loop = client.get_io_loop()
  >>> loop.run_until_complete(do_find_one())
  {'_id': ObjectId('...'), 'i': 0}

The result is a dictionary matching the one that we inserted previously.

.. note:: The returned document contains an ``"_id"``, which was
   automatically added on insert.

.. mongodoc:: find

Querying for More Than One Document
-----------------------------------
Use :meth:`~motor.motor_asyncio.AsyncIOMotorCollection.find` to query for a set of documents.
:meth:`~motor.motor_asyncio.AsyncIOMotorCollection.find` does no I/O and does not require an ``await``
expression. It merely creates an :class:`~motor.motor_asyncio.AsyncIOMotorCursor` instance. The query is
actually executed on the server when you call :meth:`~motor.motor_asyncio.AsyncIOMotorCursor.to_list`
or execute an ``async for`` loop.

To find all documents with "i" less than 5:

.. doctest:: after-inserting-2000-docs

  >>> async def do_find():
  ...     cursor = db.test_collection.find({"i": {"$lt": 5}}).sort("i")
  ...     for document in await cursor.to_list(length=100):
  ...         pprint.pprint(document)
  ...
  >>> loop = client.get_io_loop()
  >>> loop.run_until_complete(do_find())
  {'_id': ObjectId('...'), 'i': 0}
  {'_id': ObjectId('...'), 'i': 1}
  {'_id': ObjectId('...'), 'i': 2}
  {'_id': ObjectId('...'), 'i': 3}
  {'_id': ObjectId('...'), 'i': 4}

A ``length`` argument is required when you call ``to_list`` to prevent Motor
from buffering an unlimited number of documents.

``async for``
~~~~~~~~~~~~~

You can handle one document at a time in an ``async for`` loop:

.. doctest:: after-inserting-2000-docs

  >>> async def do_find():
  ...     c = db.test_collection
  ...     async for document in c.find({"i": {"$lt": 2}}):
  ...         pprint.pprint(document)
  ...
  >>> loop = client.get_io_loop()
  >>> loop.run_until_complete(do_find())
  {'_id': ObjectId('...'), 'i': 0}
  {'_id': ObjectId('...'), 'i': 1}

You can apply a sort, limit, or skip to a query before you begin iterating:

.. doctest:: after-inserting-2000-docs

  >>> async def do_find():
  ...     cursor = db.test_collection.find({"i": {"$lt": 4}})
  ...     # Modify the query before iterating
  ...     cursor.sort("i", -1).skip(1).limit(2)
  ...     async for document in cursor:
  ...         pprint.pprint(document)
  ...
  >>> loop = client.get_io_loop()
  >>> loop.run_until_complete(do_find())
  {'_id': ObjectId('...'), 'i': 2}
  {'_id': ObjectId('...'), 'i': 1}

The cursor does not actually retrieve each document from the server
individually; it gets documents efficiently in `large batches`_.

.. _`large batches`: https://mongodb.com/docs/manual/tutorial/iterate-a-cursor/#cursor-batches

Counting Documents
------------------
Use :meth:`~motor.motor_asyncio.AsyncIOMotorCollection.count_documents` to
determine the number of documents in a collection, or the number of documents
that match a query:

.. doctest:: after-inserting-2000-docs

  >>> async def do_count():
  ...     n = await db.test_collection.count_documents({})
  ...     print("%s documents in collection" % n)
  ...     n = await db.test_collection.count_documents({"i": {"$gt": 1000}})
  ...     print("%s documents where i > 1000" % n)
  ...
  >>> loop = client.get_io_loop()
  >>> loop.run_until_complete(do_count())
  2000 documents in collection
  999 documents where i > 1000

Updating Documents
------------------

:meth:`~motor.motor_asyncio.AsyncIOMotorCollection.replace_one` changes a document. It requires two
parameters: a *query* that specifies which document to replace, and a
replacement document. The query follows the same syntax as for :meth:`find` or
:meth:`find_one`. To replace a document:

.. doctest:: after-inserting-2000-docs

  >>> async def do_replace():
  ...     coll = db.test_collection
  ...     old_document = await coll.find_one({"i": 50})
  ...     print("found document: %s" % pprint.pformat(old_document))
  ...     _id = old_document["_id"]
  ...     result = await coll.replace_one({"_id": _id}, {"key": "value"})
  ...     print("replaced %s document" % result.modified_count)
  ...     new_document = await coll.find_one({"_id": _id})
  ...     print("document is now %s" % pprint.pformat(new_document))
  ...
  >>> loop = client.get_io_loop()
  >>> loop.run_until_complete(do_replace())
  found document: {'_id': ObjectId('...'), 'i': 50}
  replaced 1 document
  document is now {'_id': ObjectId('...'), 'key': 'value'}

You can see that :meth:`replace_one` replaced everything in the old document
except its ``_id`` with the new document.

Use :meth:`~motor.motor_asyncio.AsyncIOMotorCollection.update_one` with MongoDB's modifier operators to
update part of a document and leave the
rest intact. We'll find the document whose "i" is 51 and use the ``$set``
operator to set "key" to "value":

.. doctest:: after-inserting-2000-docs

  >>> async def do_update():
  ...     coll = db.test_collection
  ...     result = await coll.update_one({"i": 51}, {"$set": {"key": "value"}})
  ...     print("updated %s document" % result.modified_count)
  ...     new_document = await coll.find_one({"i": 51})
  ...     print("document is now %s" % pprint.pformat(new_document))
  ...
  >>> loop = client.get_io_loop()
  >>> loop.run_until_complete(do_update())
  updated 1 document
  document is now {'_id': ObjectId('...'), 'i': 51, 'key': 'value'}

"key" is set to "value" and "i" is still 51.

:meth:`update_one` only affects the first document it finds, you can
update all of them with :meth:`update_many`::

    await coll.update_many({'i': {'$gt': 100}},
                           {'$set': {'key': 'value'}})

.. mongodoc:: update

Deleting Documents
------------------

:meth:`~motor.motor_asyncio.AsyncIOMotorCollection.delete_one` takes a query with the same syntax as
:meth:`~motor.motor_asyncio.AsyncIOMotorCollection.find`.
:meth:`delete_one` immediately removes the first returned matching document.

.. doctest:: after-inserting-2000-docs

  >>> async def do_delete_one():
  ...     coll = db.test_collection
  ...     n = await coll.count_documents({})
  ...     print("%s documents before calling delete_one()" % n)
  ...     result = await db.test_collection.delete_one({"i": {"$gte": 1000}})
  ...     print("%s documents after" % (await coll.count_documents({})))
  ...
  >>> loop = client.get_io_loop()
  >>> loop.run_until_complete(do_delete_one())
  2000 documents before calling delete_one()
  1999 documents after

:meth:`~motor.motor_asyncio.AsyncIOMotorCollection.delete_many` takes a query with the same syntax as
:meth:`~motor.motor_asyncio.AsyncIOMotorCollection.find`.
:meth:`delete_many` immediately removes all matching documents.

.. doctest:: after-inserting-2000-docs

  >>> async def do_delete_many():
  ...     coll = db.test_collection
  ...     n = await coll.count_documents({})
  ...     print("%s documents before calling delete_many()" % n)
  ...     result = await db.test_collection.delete_many({"i": {"$gte": 1000}})
  ...     print("%s documents after" % (await coll.count_documents({})))
  ...
  >>> loop = client.get_io_loop()
  >>> loop.run_until_complete(do_delete_many())
  1999 documents before calling delete_many()
  1000 documents after

.. mongodoc:: remove

Commands
--------
All operations on MongoDB are implemented internally as commands. Run them using
the :meth:`~motor.motor_asyncio.AsyncIOMotorDatabase.command` method on
:class:`~motor.motor_asyncio.AsyncIOMotorDatabase`::

.. doctest:: after-inserting-2000-docs

  >>> from bson import SON
  >>> async def use_distinct_command():
  ...     response = await db.command(SON([("distinct", "test_collection"), ("key", "i")]))
  ...
  >>> loop = client.get_io_loop()
  >>> loop.run_until_complete(use_distinct_command())

Since the order of command parameters matters, don't use a Python dict to pass
the command's parameters. Instead, make a habit of using :class:`bson.SON`,
from the ``bson`` module included with PyMongo.

Many commands have special helper methods, such as
:meth:`~motor.motor_asyncio.AsyncIOMotorDatabase.create_collection` or
:meth:`~motor.motor_asyncio.AsyncIOMotorCollection.aggregate`, but these are just conveniences atop
the basic :meth:`command` method.

.. mongodoc:: commands

.. _example-web-application-aiohttp:

A Web Application With `aiohttp`_
---------------------------------

Let us create a web application using `aiohttp`_, a popular HTTP package for
asyncio. Install it with::

  python3 -m pip install aiohttp

We are going to make a trivial web site with two pages served from MongoDB.
To begin:

.. literalinclude:: examples/aiohttp_example.py
  :language: python3
  :start-after: setup-start
  :end-before: setup-end

The ``AsyncIOMotorClient`` constructor does not actually connect to MongoDB.
The client connects on demand, when you attempt the first operation.
We create it and assign the "test" database's handle to ``db``.

The ``setup_db`` coroutine drops the "pages" collection (plainly, this code is
for demonstration purposes), then inserts two documents. Each document's page
name is its unique id, and the "body" field is a simple HTML page. Finally,
``setup_db`` returns the database handle.

We'll use the ``setup_db`` coroutine soon. First, we need a request handler
that serves pages from the data we stored in MongoDB.

.. literalinclude:: examples/aiohttp_example.py
  :language: python3
  :start-after: handler-start
  :end-before: handler-end

We start the server by running ``setup_db`` and passing the database handle
to an :class:`aiohttp.web.Application`:

.. literalinclude:: examples/aiohttp_example.py
  :language: python3
  :start-after: main-start
  :end-before: main-end

.. warning::   It is a common mistake to create a new client object for every request; this comes at a dire performance cost.
   Create the client when your application starts and reuse that one client for the lifetime of the process.
   You can maintain the client by storing a database handle from the client on your application object, as shown in this example.

Visit ``localhost:8080/pages/page-one`` and the server responds "Hello!".
At ``localhost:8080/pages/page-two`` it responds "Goodbye." At other URLs it
returns a 404.

The complete code is in the Motor repository in ``examples/aiohttp_example.py``.

.. _aiohttp: https://aiohttp.readthedocs.io/

See also the :doc:`examples/aiohttp_gridfs_example`.

Further Reading
---------------
The handful of classes and methods introduced here are sufficient for daily
tasks. The API documentation for :class:`~motor.motor_asyncio.AsyncIOMotorClient`, :class:`~motor.motor_asyncio.AsyncIOMotorDatabase`,
:class:`~motor.motor_asyncio.AsyncIOMotorCollection`, and :class:`~motor.motor_asyncio.AsyncIOMotorCursor` provides a
reference to Motor's complete feature set.

Learning to use the MongoDB driver is just the beginning, of course. For
in-depth instruction in MongoDB itself, see `The MongoDB Manual`_.

.. _The MongoDB Manual: https://mongodb.com/docs/manual/