Tutorial
How to Query Data in MongoDB Using Python: A Beginner’s Guide
Introduction
MongoDB is a popular NoSQL database and is widely used for handling unstructured data. Python developers can efficiently query MongoDB using PyMongo, an official MongoDB Python driver. This tutorial will cover various MongoDB query techniques in Python, including:
- Retrieving documents using filters: Learn how to use
PyMongoto filter documents based on specific conditions, such as equality, inequality, and logical operators. - Querying nested fields and arrays: You will learn how to access and manipulate nested fields and arrays within MongoDB documents using PyMongo.
- Using aggregation pipelines: Use MongoDB’s aggregation framework to process and transform data in several stages, including filtering, grouping, and sorting.
- Best practices for optimizing queries: Understand how to optimize your MongoDB queries for better performance, including indexing, limiting results, and projecting specific fields.
By the end of this tutorial, you’ll be able to query MongoDB databases efficiently in Python and handle performance optimization for large datasets. We will also discuss how to use MongoDB Atlas, a fully managed cloud database service, to host your MongoDB databases in the cloud.
Prerequisites
Before querying MongoDB in Python, ensure you have the following:
-
Python 3 downloaded and installed on your system.
-
MongoDB installed and running. You can refer to this tutorial on how to install MongoDB.
-
Pymongo library installed.
-
A free MongoDB Atlas database cluster deployed for demonstration and testing purposes.
Step 1 - Install MongoDB on Ubuntu
To install MongoDB on Ubuntu, follow these steps:
Note: The below steps are run on a Ubuntu system running version 24.10. If you are running any other Ubuntu version, please consider following this official documentation on how to install MongoDB on Ubuntu.
-
Import the MongoDB public key:
-
Create the list file:
-
Reload the package database:
-
Install MongoDB Community Edition:
-
Start the MongoDB service:
-
Verify the MongoDB installation:
After completing these steps, MongoDB should be installed and running on your Ubuntu system.
Step 2 - Install PyMongo
Create a project directory
In your shell, run the following command to create a directory called pymongo-quickstart for this project:
Run the following commands to create a quickstart.py application file in the pymongo-quickstart directory:
Let’s create a virtual environment and activate it:
Now let’s install PyMongo:
You can also follow the steps mentioned in this official documentation from MongoDB on how to download and install pymongo.
Step 3 - Connecting to MongoDB Atlas in Python
Here is the folder structure after installing pymongo:
In this example, we will use MongoDB Atlas to deploy a free MongoDB cluster for demonstration purposes. MongoDB Atlas is a cloud-based MongoDB service that allows users to easily deploy, manage, and scale MongoDB clusters in the cloud. It provides a free tier, making it an ideal choice for testing and demonstration purposes.
Login to MongoDB Cloud and create a MongoDB Atlas free cluster from Atlas UI.
Note: Once your free database cluster is created, please ensure you add your current IP address of the system from which you are trying to connect with MongoDB.
Simply visit the Network Access section from the Security tab on the MongoDB Altas cluster dashboard and add the IP address.
If you don’t follow this step, then you will receive the following SSL handshake failed error when you implement the below steps to query the data.
To interact with MongoDB Atlas cluster, establish a connection using PyMongo. We will use a sample dataset provided by MongoDB Atlas.
You can find the MongoDB Atlas cluster connection string from the Overview -> Connect -> Connection Instructions tab.
Add the following lines of code inside the file /root/pymongo-quickstart/quickstart.py:
Fetch a Sample Document using find_one()
The following command retrieves a single movie document from the dataset.
Note: You need to add the lines of code to query the databse, discussed in the upcoming sections of this tutorial inside the /root/pymongo-quickstart/quickstart.py file just before the database connection is closed using client.close().
You will get the following output after running the Python file:
For more on findOne(), visit the MongoDB findOne() Example tutorial.
Retrieving Documents Using find()
The find() method retrieves documents that match a given filter.
Inside the quickstart.py file, you can add the following code to query the data:
When you run this Python file you will get the below output:
Filter Queries
In the quickstart.py file, we can add the following code to find movies released after the year 2000.
This will give the below output:
Filtering Certain Columns
In the quickstart.py file, we can add the following code to find movies released after 2000 and only display the title and IMDb rating.
In this code block, we are using the projection parameter in the find() method to specify which fields to include in the result. The _id field is excluded from the result with "_id": 0, and the title and imdb.rating fields are included with 1. The limit(10) method is used to limit the number of results to 10.
You can learn more about projections in mongodb.
Here is the output:
Querying Nested Fields and Arrays
Accessing Nested Fields
In the following code, we are querying the database to find movies with an IMDb rating greater than or equal to 8.
This code block queries the database to find movies with an IMDb rating greater than or equal to 8. It then projects only the title and IMDb rating of the movies. The limit is set to 5, so it will only return the first 5 movies that match the query.
Here is the output
Filtering Arrays
In this code snippet, we are filtering the movies that belong to the Action genre and projects only the title and IMDb rating of the movies.
Using Aggregation Pipelines for Complex Queries
Aggregation pipelines in MongoDB are a way to process data in a series of stages. Each stage transforms the documents as they pass through the pipeline. This allows for more complex and powerful queries than simple find operations.
This code block uses MongoDB’s aggregation pipeline to process data in a series of stages.
The pipeline consists of two stages:
- The “$match” stage filters the documents to only include those where the “year” field is greater than or equal to 2010.
- The “$group” stage groups the filtered documents by their “genres” field and calculates the average “imdb.rating” for each group.
The result of the aggregation pipeline is stored in the “result” variable.
Finally, the code iterates over the “result” and prints each document, which represents the average IMDb rating for each genre of movies released in or after 2010.
Here is the output when this query is run:
Differences between find() and aggregate() Methods
| Feature | find() |
aggregate() |
Sample Example |
|---|---|---|---|
| Simplicity | Simple | Complex | db.collection.find({}) vs db.collection.aggregate([{ $match: {} }]) |
| Performance | Fast for simple queries | Optimized for large datasets | db.collection.find({ year: { $gte: 2010 } }) vs db.collection.aggregate([{ $match: { year: { $gte: 2010 } } }, { $group: { _id: "$genres", averageIMDB: { $avg: "$imdb.rating" } } }]) |
| Use Case | Filtering & retrieving data | Data transformation & computation | db.collection.find({ genres: "Comedy" }) vs db.collection.aggregate([{ $match: { genres: "Comedy" } }, { $group: { _id: "$genres", count: { $sum: 1 } } }]) |
The find() method is a straightforward way to retrieve data from a MongoDB collection. It is ideal for simple queries that require filtering and retrieving data based on specific conditions. However, it has limitations when it comes to complex data transformations and computations.
On the other hand, the aggregate() method is a powerful tool for processing data in a series of stages. It is designed to handle complex data transformations, aggregations, and computations. While it can be more complex to use than find(), it offers a wide range of operators and stages that can be combined to perform sophisticated data processing tasks.
When deciding between find() and aggregate(), consider the complexity of your query and the type of data processing required. If you need to perform simple filtering and retrieval, find() might be the better choice. However, if you need to perform data transformations, aggregations, or computations, aggregate() is likely the better option.
Optimizing MongoDB Queries for Performance
Use Indexes
Creating an index on a field can significantly improve query performance, especially for fields used in query filters or sorts. In this example, we create an index on the “year” field to optimize queries that filter movies by release year.
Limit Results
Limiting the number of results returned by a query can reduce the amount of data transferred and processed, leading to improved performance. This is particularly useful when you only need a subset of the data or want to implement pagination. Here, we limit the results to the first 10 movies found.
Project Specific Fields
Projecting specific fields in a query allows you to retrieve only the data you need, reducing the amount of data transferred and processed. This can improve performance and reduce network overhead. In this example, we project only the “title” field and exclude the “_id” field to only display the titles of the movies.
FAQs
1. How do I query data in MongoDB using Python?
When querying data in MongoDB using Python, you have two primary methods: find() and aggregate(). The choice between these methods depends on the complexity of your query and the type of data processing required.
Basic Queries with find()
The find() method is suitable for simple queries that require filtering and retrieving data based on specific conditions. It is ideal for tasks such as:
- Retrieving all documents in a collection:
db.collection.find() - Filtering documents based on a condition:
db.collection.find({"year": 2010}) - Retrieving specific fields:
db.collection.find({}, {"title": 1, "_id": 0})
Here’s an example of using find() to retrieve all movies released in 2010:
Advanced Queries with aggregate()
The aggregate() method is designed for complex data transformations, aggregations, and computations. It is ideal for tasks such as:
- Grouping documents based on a field and calculating an average:
db.collection.aggregate([{ $group: { _id: "$genres", averageIMDB: { $avg: "$imdb.rating" } } }]) - Filtering documents based on a condition and then grouping:
db.collection.aggregate([{ $match: { year: 2010 } }, { $group: { _id: "$genres", count: { $sum: 1 } } }])
Here’s an example of using aggregate() to calculate the average IMDB rating for movies in each genre:
In summary, use find() for basic queries that require simple filtering and retrieval, and use aggregate() for advanced queries that require complex data transformations and computations.
2. What is the difference between find() and aggregate()?
The find() method is a straightforward way to retrieve data from a MongoDB collection. It is ideal for simple queries that require filtering and retrieving data based on specific conditions. However, it has limitations when it comes to complex data transformations and computations.
On the other hand, the aggregate() method is a powerful tool for processing data in a series of stages. It is designed to handle complex data transformations, aggregations, and computations. While it can be more complex to use than find(), it offers a wide range of operators and stages that can be combined to perform sophisticated data processing tasks.
3. How do I filter data in MongoDB with Python?
To filter data in MongoDB using Python, you can use the { "field": { "$operator": value } } syntax. This syntax allows you to specify a field, an operator, and a value to filter the data. Here’s an example code block that demonstrates how to filter documents where the “year” field is equal to 2020:
4. Can I query nested documents in MongoDB?
Yes, you can query nested documents in MongoDB using dot notation. Dot notation allows you to access fields within nested documents. The syntax for dot notation is { "nested.field": value }.
For example, if you have a document structure like this:
You can query documents where the “city” field within the “address” nested document is “Anytown” using the following code:
This will return all documents where the “city” field within the “address” nested document matches “Anytown”.
5. How do I optimize MongoDB queries for large datasets?
You can use techniques like indexing, projection, and limit results to improve performance as discussed in the sections above.
6. What are the best libraries for MongoDB querying in Python?
-
pymongo(official MongoDB driver) -
mongoengine(object-document mapper)
For more MongoDB management tutorials, check out our tutorial series on How to Manage Data with MongoDB.
Conclusion
In conclusion, querying MongoDB in Python is a crucial skill for any developer working with large datasets. This tutorial has covered various MongoDB query techniques in Python, including retrieving documents using filters, querying nested fields and arrays, using aggregation pipelines, and best practices for optimizing queries.
In this tutorial you learned:
- Crafting basic queries using the
find()method to retrieve specific documents - Filtering nested fields and arrays to target specific data and reduce the amount of data retrieved
- Leveraging aggregation pipelines using
aggregate()for complex data processing, such as data transformation, grouping, and sorting - Optimizing queries for performance on large datasets by using indexes, projections, and limits to reduce the load on the database and improve query efficiency
By mastering these techniques, you’ll be able to efficiently retrieve and manipulate data in your MongoDB collections, making you a more effective developer in handling large datasets.
For further reading, explore:
-
How to Create Queries in MongoDB tutorial series.
Continue building with DigitalOcean GenAI Platform.
About the author(s)
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Become a contributor for community
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
DigitalOcean Documentation
Full documentation for every DigitalOcean product.
Resources for startups and SMBs
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
The developer cloud
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Get started for free
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.