Finding Documents with Filters

A filter object tells find and findOne which documents you want, with operators such as $gte and $in for comparisons, dot notation for a field inside a field, an ObjectId to find one document by its id, and a projection to send back only the fields you need.

Apprentice · MongoDB · NoSQL · Node.js · Updated

By

After this lesson you can

  • Find documents with find and findOne and a filter, using $gt, $gte, $lt, $lte, $in and dot notation
  • Find one document by its id, turning the id text into an ObjectId and handling an id that isn't valid
  • Choose the fields a query sends back with a projection

Key takeaways

  • find(filter) gives back every document that matches the filter, and findOne(filter) gives back the first one, or null when nothing matches.
  • Every field in a filter must match. A plain value for an array field matches any document whose array holds that value.
  • Operators compare instead of matching exactly. $gt means greater than, $gte greater than or equal to, $lt less than, $lte less than or equal to, and $in any value in a list.
  • An id arrives as text, so check it with ObjectId.isValid(id), then turn it into an ObjectId with new ObjectId(id) before you search for it.
  • A projection picks the fields that come back. 1 keeps a field and 0 leaves it out, and _id comes along unless it is marked 0.

A shopper opens a market's app and taps Organic only, or types honey into the search box. The app asks your server, and your server asks MongoDB. Nearly every screen of a real app is a question like that, so asking MongoDB for exactly the documents you need is the database code you'll write most.

This chapter's example is the Saturday Farmers Market. Its twelve vendors each have a name, a stall number, the products they sell, whether they're organic, a rating from shoppers, and the farm they come from.

The market's data, and the slow way to search

Make a folder named market with a package.json holding { "type": "module" }, run npm install mongodb, and add the .env file from Connecting from Node and Saving Documents, which holds MONGODB_URI=mongodb://localhost:27017. Then save this as seed-market.js. It empties the vendors collection first, so it's safe to run again whenever you want the data back as it started:

import { MongoClient } from "mongodb";

const client = new MongoClient(process.env.MONGODB_URI);
const vendors = client.db("market").collection("vendors");

await vendors.deleteMany({});
const result = await vendors.insertMany([
  { name: "Little Red Hen Eggs", stall: 1, products: ["eggs"], organic: true, rating: 3.8, farm: { town: "Maple Hollow", miles: 27 } },
  { name: "Green Valley Greens", stall: 2, products: ["lettuce", "kale", "spinach"], organic: true, rating: 4.2, farm: { town: "Willow Creek", miles: 8 } },
  { name: "Petal & Stem", stall: 3, products: ["tulips", "sunflowers"], organic: false, rating: 3.9, farm: { town: "Willow Creek", miles: 6 } },
  { name: "Sunny Acres", stall: 4, products: ["tomatoes", "peppers", "zucchini"], organic: true, rating: 4.7, farm: { town: "Willow Creek", miles: 12 } },
  { name: "Berry Patch", stall: 5, products: ["strawberries", "blueberries", "jam"], organic: true, rating: 4.8, farm: { town: "Cedar Ridge", miles: 18 } },
  { name: "Herb Corner", stall: 6, products: ["basil", "mint", "rosemary"], organic: true, rating: 4.0, farm: { town: "Cedar Ridge", miles: 15 } },
  { name: "Orchard Lane", stall: 7, products: ["apples", "pears", "cider"], organic: false, rating: 4.4, farm: { town: "Cedar Ridge", miles: 22 } },
  { name: "Maple Grove Syrup", stall: 8, products: ["maple syrup", "honey"], organic: false, rating: 4.3, farm: { town: "Maple Hollow", miles: 29 } },
  { name: "Bee Happy Honey", stall: 9, products: ["honey", "beeswax candles"], organic: false, rating: 4.9, farm: { town: "Maple Hollow", miles: 31 } },
  { name: "Fungi Friends", stall: 10, products: ["mushrooms"], organic: true, rating: 4.1, farm: { town: "Stone Bridge", miles: 35 } },
  { name: "Pumpkin Patch", stall: 11, products: ["pumpkins", "squash"], organic: true, rating: 4.5, farm: { town: "Stone Bridge", miles: 44 } },
  { name: "Hilltop Dairy", stall: 12, products: ["cheese", "yogurt", "milk"], organic: false, rating: 4.6, farm: { town: "Stone Bridge", miles: 40 } },
]);
console.log(`Added ${result.insertedCount} vendors.`);

await client.close();
node --env-file=.env seed-market.js
Added 12 vendors.

Now, who sells honey? The first idea is the one you'd use on an array: read every vendor, then keep the ones you want. Save this as find.js:

import { MongoClient, ObjectId } from "mongodb";

const client = new MongoClient(process.env.MONGODB_URI);
const vendors = client.db("market").collection("vendors");

const all = await vendors.find().toArray();
const honey = all.filter((vendor) => vendor.products.includes("honey"));
console.log(`Read ${all.length} vendors to keep ${honey.length}.`);

await client.close();
Read 12 vendors to keep 2.

It works, with twelve vendors. With 5,000, every search would send all 5,000 documents from the database to your program to keep two. It's better to hand MongoDB the question and get back only the answer.

A filter: asking MongoDB for less

At the market's information booth, you don't ask for every stall's whole story. You say what you're looking for: "the organic ones". In MongoDB, that description is a filter: an object listing the values the documents must have. Hand it to find.

Each example from here on replaces the middle of find.js, the lines between const vendors and await client.close():

const organic = await vendors.find({ organic: true }).toArray();
console.log(organic.map((vendor) => vendor.name));

const honey = await vendors.find({ products: "honey" }).toArray();
console.log(honey.map((vendor) => vendor.name));

const stall4 = await vendors.findOne({ stall: 4 });
console.log(stall4.name);

console.log(await vendors.findOne({ stall: 40 }));
[
  'Little Red Hen Eggs',
  'Green Valley Greens',
  'Sunny Acres',
  'Berry Patch',
  'Herb Corner',
  'Fungi Friends',
  'Pumpkin Patch'
]
[ 'Maple Grove Syrup', 'Bee Happy Honey' ]
Sunny Acres
null
  • find(filter) finds every match, and toArray() collects them into an array.
  • products is an array, and { products: "honey" } matches any vendor whose array holds "honey".
  • findOne(filter) gives back the first match itself, not an array, or null when nothing matches. There's no stall 40.

Comparing with $gte, $lt and $in

Shoppers don't always want an exact value: "rated 4.5 or better", "grown less than 10 miles away". For those, the field's value in the filter is an object holding a query operator, a name that starts with $ (read "dollar"):

Operator Read it Matches
$gt greater than values above it
$gte greater than or equal to values at or above it
$lt less than values below it
$lte less than or equal to values at or below it
$in in any value in the list
const topRated = await vendors.find({ rating: { $gte: 4.5 } }).toArray();
console.log(topRated.map((vendor) => `${vendor.name} ${vendor.rating}`));

const organicTop = await vendors.find({ organic: true, rating: { $gte: 4.5 } }).toArray();
console.log(organicTop.map((vendor) => vendor.name));

const sweet = await vendors.find({ products: { $in: ["honey", "jam"] } }).toArray();
console.log(sweet.map((vendor) => vendor.name));

const nearby = await vendors.find({ "farm.miles": { $lt: 10 } }).toArray();
console.log(nearby.map((vendor) => `${vendor.name}, ${vendor.farm.miles} miles`));
[
  'Sunny Acres 4.7',
  'Berry Patch 4.8',
  'Bee Happy Honey 4.9',
  'Pumpkin Patch 4.5',
  'Hilltop Dairy 4.6'
]
[ 'Sunny Acres', 'Berry Patch', 'Pumpkin Patch' ]
[ 'Berry Patch', 'Maple Grove Syrup', 'Bee Happy Honey' ]
[ 'Green Valley Greens, 8 miles', 'Petal & Stem, 6 miles' ]

Two fields in one filter must both match, so the second search finds organic vendors rated 4.5 or more. Two operators on one field go in the same object: { stall: { $gt: 3, $lt: 7 } } finds stalls 4, 5 and 6.

The last search reaches into a field inside a field with dot notation: "farm.miles", read "farm dot miles", is the miles inside farm. It needs quotes, because a dot isn't allowed in a plain JavaScript property name.

One vendor by its id

A vendor's page in the app has an address such as /vendors/6afc92868a3ab51b4f41bdd3, so its id reaches your code as text. But _id is stored as an ObjectId, and text never equals an ObjectId. new ObjectId(id) turns the text into one. Text that isn't 24 hexadecimal characters (the digits 0 to 9 and the letters a to f) stops the program with BSONError: input must be a 24 character hex string, 12 byte Uint8Array, or an integer, so check it first with ObjectId.isValid(id):

async function findVendor(id) {
  if (!ObjectId.isValid(id)) {
    return null;
  }
  return vendors.findOne({ _id: new ObjectId(id) });
}

const sunny = await vendors.findOne({ name: "Sunny Acres" });
const id = sunny._id.toString();

console.log(await vendors.findOne({ _id: id }));
console.log((await findVendor(id)).name);
console.log(await findVendor("stall-4"));
null
Sunny Acres
null

The first search, with the id as text, finds nothing. findVendor finds Sunny Acres with the ObjectId, and gives back null for "stall-4", the same answer as an id that matches no vendor, instead of crashing. In the next chapter, that null becomes a 404.

Only the fields you need: projection

A list of stalls needs each vendor's name and stall, not its farm and products. A projection, in find's second argument, picks the fields: 1 keeps a field, and 0 leaves it out. _id comes along unless you mark it 0:

const listing = await vendors
  .find({ organic: true }, { projection: { _id: 0, name: 1, stall: 1 } })
  .toArray();
console.log(listing);
[
  { name: 'Little Red Hen Eggs', stall: 1 },
  { name: 'Green Valley Greens', stall: 2 },
  { name: 'Sunny Acres', stall: 4 },
  { name: 'Berry Patch', stall: 5 },
  { name: 'Herb Corner', stall: 6 },
  { name: 'Fungi Friends', stall: 10 },
  { name: 'Pumpkin Patch', stall: 11 }
]

Use one way or the other: list the fields to keep, or list the fields to leave out ({ projection: { farm: 0 } }). Mixing them, apart from _id, stops with MongoServerError: Cannot do exclusion on field farm in inclusion projection.

Check yourself

2 questions to try on the lesson page.

So what

A filter hands MongoDB the question and gets back only the answer, so your API stays fast when the market has 5,000 vendors, and every search, list and "one item by id" route you'll build rests on these three ideas.

Key terms

Related