MongoDB — Documents, Sharding, and the NoSQL Default

If you always read the order and its items in the same request, embed the items array inside the order document. One read, no JOIN, fast.

Don't embed a user's entire comment history inside the user document — a power user might have 100,000 comments, and MongoDB documents have a 16 MB cap. Store comments in a separate collection, reference the user by user_id.

A write that touches one document is guaranteed atomic — either the whole update lands or none of it does. This is free, built-in, and requires no transaction syntax. Design your documents so that the operations you care about fit inside one document.

MongoDB lets you enforce shapes using $jsonSchema validators at the collection level. You get the flexibility to evolve your schema over time while still catching obviously wrong inserts. Use validators in production; skip them in early prototypes.

The top-level unit of storage — like a row, but with a flexible shape. A document is a set of key-value pairs that looks like JSON. It can contain strings, numbers, arrays, nested objects, dates, and special types like ObjectId and Binary. Every document in MongoDB has a special field called _id that uniquely identifies it within its collection.

{ "_id": ObjectId("..."),
  "name": "Alice",
  "tags": ["admin", "user"],
  "address": { "city": "London" }
}

A named group of related documents — like a table, but without a fixed schema. You create a collection by inserting your first document into it. All documents in a collection live on the same set of shards and share the same indexes. Within one collection, documents can have completely different shapes (though in practice you usually keep them similar).

Binary JSON — the wire format and on-disk format MongoDB uses instead of plain JSON text. BSON adds types that JSON doesn't have: ObjectId, Date, Decimal128, Binary, and Int32 / Int64 (JSON has only "number"). BSON is typically faster to parse than JSON because the byte layout encodes field lengths upfront, so the parser can skip fields without scanning the whole document. It's also more compact for typed numeric data.

A group of MongoDB nodes that hold the same data — one primary accepts all writes, one or more secondaries replicate from the primary asynchronously. If the primary goes down, the secondaries hold an automatic election and promote one of themselves. This gives you high availability with no manual intervention. A replica set also keeps an oplog — an ordered log of every write, which secondaries use to stay in sync (and which is also the foundation of change streams).

The architecture for horizontal scale. A collection is split into chunks (contiguous ranges of shard key values). Each chunk lives on one shard — and each shard is itself a replica set. A lightweight process called mongos acts as a query router: your application talks to mongos exactly as it would talk to a regular MongoDB node; mongos figures out which shard(s) have the data and forwards the query. Config servers hold the chunk map (which shard owns which range).

A way to transform data on the server before it reaches your application. An aggregation pipeline is a sequence of stages — each stage takes documents in, transforms or filters them, and passes results to the next stage. Common stages: $match (filter), $group (aggregate like SQL GROUP BY), $sort, $limit, $project (reshape), $lookup (JOIN from another collection), $unwind (flatten an array into separate documents). This is how MongoDB does analytics and reporting without pulling raw data into your app layer.

The default. Create one on any field you query frequently by equality or range. Supports both ascending and descending scans (direction only matters for sort performance on compound indexes).

db.users.createIndex({ email: 1 })
// 1 = ascending, -1 = descending

Covers queries on multiple fields together. Follows the same left-prefix rule as SQL composite indexes: an index on (a, b, c) can satisfy queries on a, a + b, or a + b + c — but NOT a query on b alone.

db.orders.createIndex({ status: 1, created_at: -1 })
// Satisfies: { status: "shipped" } sorted by date

Automatically created when you index a field that holds an array. MongoDB creates one index entry per array element, so you can query { tags: "electronics" } and it will hit the index — even though tags is an array. One limitation: you can't create a compound multikey index if both indexed fields are arrays.

db.products.createIndex({ tags: 1 })
// { tags: ["electronics","wireless"] }
// → creates entries for BOTH "electronics" AND "wireless"

Powers full-text search across string fields. Tokenizes words, strips stop words, and supports phrase search and word-stem matching. A collection can have only one text index (but it can cover multiple fields). For large-scale search needs, many teams move to dedicated tools like Elasticsearch — the text index covers basic search well enough for moderate traffic.

db.articles.createIndex({ title: "text", body: "text" })
db.articles.find({ $text: { $search: "machine learning" } })

For "find things near me" queries. Indexes GeoJSON geometry fields (Points, LineStrings, Polygons). Supports $near (closest N), $geoWithin (inside a polygon), and $geoIntersects. Uses an S2 spherical geometry library, so distance calculations are accurate across the Earth's curvature.

db.places.createIndex({ location: "2dsphere" })
db.places.find({
  location: {
    $near: {
      $geometry: { type: "Point", coordinates: [-0.127, 51.507] },
      $maxDistance: 1000  // metres
    }
  }
})

Automatically deletes documents after a set number of seconds. Perfect for sessions, cache entries, temporary tokens, or log data you only need for N days. MongoDB runs a background task roughly every 60 seconds to expire documents — so deletion isn't instantaneous, but it's close enough for most use cases.

// Sessions expire 30 minutes after last_active
db.sessions.createIndex(
  { last_active: 1 },
  { expireAfterSeconds: 1800 }
)
// MongoDB deletes documents where
// Date.now() - last_active > 1800 seconds

The oplog (operation log) is a special capped collection on the primary that records every write in order — think of it as a changelog for the database. Secondaries tail the oplog just like a Unix tail -f and replay each entry to stay in sync. Because it's capped, older entries roll off after a while. The oplog window (how far back it holds history) is typically hours to days on a healthy cluster — important for recovering a secondary that was offline briefly.

When the primary goes silent, secondaries hold an automatic election using a Raft-like protocol. Each node votes for the candidate with the most up-to-date oplog. The winner needs a majority of votes — that's why you want an odd number of nodes (3, 5, 7). With 2 nodes, neither can ever reach majority alone; with 3 nodes, the 2 survivors always have majority. An "arbiter" node counts for voting but holds no data — it's a cheap way to break ties.

By default, clients read from the primary. But you can dial this. primaryPreferred reads from primary but falls back to a secondary if the primary is unreachable. secondary always reads from a secondary — useful for analytics queries you don't want competing with writes. nearest picks the node with the lowest network latency regardless of role. The trade-off: secondaries can be slightly behind (eventual consistency), so use primary when you need the absolute latest value.

Write concern tells MongoDB how many nodes must acknowledge a write before calling it "done." w: 1 means just the primary confirmed it — fastest, but data could be lost if the primary crashes before replicating. w: majority waits for a majority of nodes to confirm — safe against primary failure, slightly slower. w: 3 waits for exactly 3 nodes. For financial data or any write you can't afford to lose, always use w: majority.

Documents are assigned to shards based on ranges of the shard key value. For example, user IDs 1–10,000 go to Shard 1, 10,001–20,000 go to Shard 2, and so on. The upside is that range queries (user_id: { $gte: 5000, $lte: 8000 }) can be routed to a single shard — no fan-out. The downside is hotspots: if most of your new writes have high user IDs (monotonically increasing), they'll all pile onto the last shard.

MongoDB hashes the shard key value and uses the hash to assign the document to a shard. This distributes documents evenly at random — no hotspots, perfect write balance. The cost: range queries now scatter across all shards because adjacent key values hash to different locations. Best for insert-heavy workloads where point lookups are more common than ranges.

You pin specific shard key ranges to specific shards using zones. Example: all documents where region: "EU" must stay on EU-located shards for GDPR compliance. Or: your "hot" products (high view counts) stay on high-IOPS SSDs while archival data lives on cheaper spinning disks. It's the escape hatch when you need data locality for legal, latency, or cost reasons.

Old MMAPv1 used a collection-level lock — if one writer was updating any document, all other writers to that collection had to wait. WiredTiger lets each writer see its own consistent view of the data without blocking other writers — a technique called MVCC (multi-version concurrency control). Writers touching different documents never block each other, which is why MongoDB handles high-concurrency write workloads on the same collection without writers queuing up.

WiredTiger compresses data pages before writing to disk using snappy by default — a compression algorithm tuned for speed over ratio, adding very little CPU overhead. Switching to zstd typically saves 30–50 % more space at the cost of slightly more CPU. For index pages, the default is no compression (indexes are already structured data; compressing them rarely saves much). Compression means less disk I/O, which means faster reads from spinning disks and lower storage bills in cloud environments.

Before a write is applied to the B+ tree in the buffer cache, WiredTiger first appends it to the journal — a sequential append-only log file on disk. If the server crashes before the checkpoint, MongoDB replays the journal on startup to recover any writes that hadn't been checkpointed. The journal syncs to disk roughly every 100 ms (configurable). This is what guarantees no data loss on crash with default settings — the write-ahead log is the industry-standard durability pattern used by Postgres, MySQL, and others.

Every 60 seconds by default, WiredTiger writes all dirty (modified) pages from the buffer cache to the B+ tree files on disk and advances the "checkpoint" marker. Once a checkpoint succeeds, the journal entries up to that point can be discarded — they've been safely persisted. Between checkpoints, only the journal guarantees durability. The 60-second interval is a balance: shorter means less journal replay on crash, longer means fewer write amplification spikes during the checkpoint flush.

Like a SQL WHERE clause. Uses indexes when placed first in the pipeline. Always put $match as early as possible — it reduces the number of documents flowing into subsequent expensive stages. A $match late in the pipeline is wasteful because every earlier stage processed documents that were just going to be thrown away.

Include, exclude, or rename fields. Like SQL's SELECT a, b AS alias. Use it to strip out large fields you don't need downstream — less data flowing between stages means faster pipelines.

The SQL GROUP BY equivalent. Group documents by a key and compute accumulators: $sum, $avg, $min, $max, $count, $push (collect into array). Example: total revenue per product category.

Left outer join from another collection. Slower than a SQL JOIN (documents aren't pre-joined on disk), but fully functional. Works best when the joined collection is small or well-indexed. For hot paths, design schemas to avoid $lookup entirely by embedding.

Deconstructs an array field: a document with a 3-element array becomes 3 documents, each with one element. Essential when you need to aggregate on individual array values rather than the whole array.

Order results, take top N, or skip N for pagination. Putting $sort + $limit together is optimized by MongoDB — it doesn't need to sort the entire result set if you only want the top 10.

Run multiple sub-pipelines on the same input in one pass. Classic use case: a search results page that simultaneously returns the matching items and the facet counts (how many results per category, per price range, etc.) without two separate queries.

The default isolation level inside a MongoDB transaction is snapshot — when your transaction starts, it sees a consistent point-in-time view of all data. Concurrent writers don't affect what you read. This prevents dirty reads, non-repeatable reads, and phantom reads — the classic isolation bugs. It's the same isolation level Postgres uses by default.

Single-shard transactions are fast. Cross-shard transactions require a two-phase commit across coordinator nodes — dramatically more round trips, locks held longer, higher latency. If your transaction touches data on 4 different shards, every operation in it has to coordinate across all 4. This is why good shard key design matters: keep data that belongs to the same transaction on the same shard.

A transaction that hasn't committed within 60 seconds is automatically aborted. This prevents abandoned transactions from holding locks indefinitely. The 60-second limit is configurable via transactionLifetimeLimitSeconds — but the right answer is usually to shorten transactions, not lengthen the timeout. Keep transactions small and fast.

The fact that you can use multi-document transactions doesn't mean you should restructure your whole application around them. The MongoDB team's advice: design your schema so that operations you care about happen inside a single document. Transactions exist for the cases where that's genuinely not possible — transfers, order + inventory decrement, etc. Single-document atomicity is free; transactions have overhead.

Keep related data inside the parent document as a nested array. An order with its line items, a blog post with its three tags, a user with two addresses — these are all "few" relationships. The payoff is a single read that returns everything at once. No join, no second query.

When to embed: the child data is always needed with the parent, the list stays small (under a few hundred items), and the child data has no independent lifecycle.

Store the child in a separate collection and put only a reference (like user_id) in the parent — or vice versa. A user with thousands of orders, a post with hundreds of comments. Embedding these would balloon the parent document, eventually hitting MongoDB's 16 MB per-document cap. References keep the parent document lean and let you query children independently.

Instead of one document per event (a sensor reading every second = 86,400 documents/day), group events into time windows — one document per minute or hour with an array of readings. This dramatically reduces document count, makes range reads faster (one I/O for an hour's data), and shrinks index size. It's the go-to pattern when you're storing a steady stream of events tagged with timestamps — sensor readings, app metrics, audit logs — the kind of workload the industry calls time-series.

Most documents in a collection are "normal" — but a small number are exceptional. A celebrity Twitter account has millions of followers; an average user has 200. If you embed the followers array, the celebrity's document could be gigabytes. The outlier pattern: store the first N entries inline, then set a flag has_overflow: true and write the rest to an overflow collection. Normal documents stay fast; outliers are handled gracefully.

Pre-calculate expensive aggregations at write time and store the result on the document. A blog post's comment_count, a product's average_rating, a page's view_count. Reading the count is now a single field fetch — not a COUNT(*) aggregation across thousands of rows. You trade a small write overhead (increment the counter on every new comment) for dramatically cheaper reads. Works beautifully for read-heavy metrics.

Every stage after $match only processes the documents that survive the filter. If your match reduces 1 million documents to 500, everything downstream runs on 500 documents — a 2,000x reduction in work. MongoDB can even push the $match filter into an index scan automatically if you place it first.

A $sort on a large result set is expensive because it must hold all the data in memory at once. If you sort on an indexed field and place $sort right after $match, MongoDB may be able to do an index-ordered scan — returning documents already in sorted order, with no in-memory sort at all. Check with explain() whether the sort stage says SORT (bad, in-memory) or is absorbed into the index scan.

If later stages only need five fields from a 40-field document, add a $project early to drop the unused fields. Smaller documents flow through subsequent stages faster, use less memory, and transfer less data between shards on a sharded cluster.

$lookup is MongoDB's equivalent of a JOIN — for each input document it runs a separate query against another collection. At scale this is expensive. If you're running $lookup against a large collection on a frequently-run pipeline, consider embedding the data instead (Schema Design: Embedded One-to-Few). If you must use $lookup, always have an index on the joined field in the foreign collection.

By default, an aggregation pipeline that needs more than 100 MB of memory for a sort or group stage will just error out. Adding allowDiskUse: true lets MongoDB spill to disk. This prevents crashes but makes the query significantly slower. Treat it as a safety net, not a performance strategy — if you hit the 100 MB limit regularly, fix the pipeline first.

Never guess whether your pipeline is fast — check. Append .explain("executionStats") to any aggregation to see the execution plan: which indexes were used, how many documents were examined at each stage, and how long each stage took. It's the fastest way to catch a missing index or a mis-ordered pipeline before it hurts production.

mongodump is a logical backup tool — it reads documents and writes them to BSON files. It works fine for small datasets but gets slower as data grows because it reads every document. For production, filesystem snapshots (LVM, EBS snapshots on AWS, or Atlas's continuous backup) are faster and more reliable — they capture the storage layer directly in seconds. Ops Manager and Atlas both offer point-in-time restore. Whatever you use, test restores regularly — a backup you've never restored from might not actually work.

Four metrics deserve alerts: replica lag (if it exceeds the oplog window a secondary must full-resync), oplog window size (keep it above 24 hours so a brief secondary outage doesn't force a full resync), WiredTiger cache hit rate (below ~90% means the working set no longer fits in RAM — add memory or reduce hot data size), and slow query log (any query over 100ms likely needs an index). Free tools: mongostat, mongotop; commercial: Atlas, Datadog, Grafana with the MongoDB Exporter.

Replica sets support rolling upgrades — upgrade one member at a time (secondaries first, then the primary) so the cluster stays available throughout. Major version jumps (e.g. 5.0 → 6.0) require an extra step: you must first raise the Feature Compatibility Version (FCV) to the current version before upgrading. Skipping that step can leave the cluster in an inconsistent FCV state that's hard to recover from. Always read the release notes before upgrading.

Versions of MongoDB before 3.0 shipped with authentication disabled by default. Many developers installed MongoDB for local development, accidentally exposed it to the internet, and had databases wiped and held for ransom. Always explicitly enable authentication (--auth flag or security.authorization: enabled in config). Modern versions enable auth by default when using the package manager on most distros — but double-check your config file. Use SCRAM-SHA-256 for user auth. Restrict network access: bind MongoDB to specific IPs, not 0.0.0.0.

In transit: enable TLS between clients and mongod, and between replica set members. Without TLS, credentials and data travel in plaintext over the network. At rest: WiredTiger supports an encrypted storage engine (available in MongoDB Enterprise and Atlas). For regulated industries (healthcare, finance) where compliance requires encryption at rest, this is non-negotiable. Client-side field-level encryption (CSFLE) goes further — data is encrypted before it even leaves your application, so even a DB admin can't read sensitive fields.

MongoDB drivers maintain a connection pool — a set of pre-opened connections that requests share instead of opening a new TCP connection per query. Opening a TCP connection + TLS handshake can take 10-50 ms; reusing a pooled connection takes under 1 ms. The default maxPoolSize in most drivers is 100. For high-concurrency services (>100 simultaneous requests), tune this up. For serverless functions that spin up many instances, tune it down — each function instance creates its own pool, and 1,000 Lambda instances × 100 connections each = 100,000 connections which will exhaust mongod's connection limit.

Product catalogs, content management systems, user profile stores with extension fields — any app where the "shape" of a record varies significantly per record type. Instead of a wide sparse table with hundreds of NULL columns or a JSONB blob that the database can't index efficiently, each document carries exactly its own fields. Adding a new product category tomorrow requires zero schema migration.

When write volume exceeds what a single server can handle, MongoDB's built-in sharding distributes writes across multiple shard replica sets automatically. This is substantially easier than horizontal write scaling in Postgres (which requires external tooling like Citus, or a manual application-level sharding strategy). IoT event ingestion, mobile analytics, and high-throughput behavioral event stores are common use cases.

A bank transfer deducting from account A and crediting account B needs strict ACID across two separate entities. MongoDB does support multi-document transactions (since version 4.0), but they carry overhead and the programming model is more awkward than Postgres's. For applications where most operations are multi-entity ACID transactions, Postgres is simply a better tool. The relational model was designed for exactly this.

If your team writes complex analytical queries — multi-table joins, window functions, CTEs, GROUP BY rollups across the whole dataset — Postgres is significantly more comfortable. The aggregation pipeline can do some of this, but it's more verbose than SQL and lacks features like CTEs. For serious analytics, most teams pair MongoDB with a separate data warehouse (BigQuery, Redshift, Snowflake) and use dbt-style transformations on exported data.

Foreign key constraints, cascading deletes, check constraints enforced at the database layer — MongoDB has none of these. You can implement referential integrity in application code, but it's error-prone under concurrency and not enforced at the database layer. If your app has 15 entities with complex many-to-many relationships and strict integrity requirements, a relational database is a much better fit.

Best of both worlds within one engine. A JSONB column sits next to normal typed columns in the same table. You can join a JSONB field to another table's integer ID, run window functions over a mix of relational and JSON data, and wrap everything in a single ACID transaction. GIN indexes on JSONB let you query nested keys efficiently. The tooling ecosystem (pgAdmin, DBeaver, dbt, every analytics BI tool) works out of the box.

Limitation: JSONB doesn't do horizontal write sharding natively. Past a certain write volume, you need Citus or application-level sharding. Also, JSONB fields don't get the rich type system that BSON provides (no ObjectId, no Date type — just strings in JSON).

Document-first from top to bottom. The driver, the query language, the aggregation pipeline, the index types (multikey, geospatial, text, TTL) — all designed around documents. Horizontal sharding is built in and works transparently to the application. BSON gives richer types than JSON. Change streams let you subscribe to real-time document updates without polling. Atlas extends the open-source base with Lucene-based full-text search and vector search for AI/ML workloads.

Limitation: weaker SQL-style analytics, no native FK constraints, more awkward for heavily relational workloads. Aggregation pipeline syntax is more verbose than SQL for complex transformations.

The official cloud service from MongoDB, Inc. — this is what most new projects use in 2024. Atlas runs the actual MongoDB engine on your choice of AWS, Google Cloud, or Azure. It handles backups (continuous and point-in-time restore), monitoring with built-in alerts, automatic minor version upgrades, and a visual performance advisor that suggests indexes based on your actual query patterns.

Atlas extends beyond the open-source release in two notable ways: Atlas Search (a Lucene-based full-text search engine built directly into the cluster — no separate Elasticsearch needed) and Atlas Vector Search (stores and queries embedding vectors for AI/ML similarity search, without needing a dedicated vector database). These features are becoming core to many production workloads.

Amazon's MongoDB-compatible database service. It exposes a MongoDB wire protocol API — meaning drivers and most application code work without changes — but the underlying engine is completely different from MongoDB. Amazon built their own storage engine that understands MongoDB's API but operates differently internally.

This distinction has real consequences: performance characteristics differ, some features are missing or behave differently (change streams, certain aggregation operators, $lookup behavior), and you may hit compatibility issues with newer MongoDB driver features. It can be a reasonable choice if you're deeply committed to the AWS ecosystem and want to avoid MongoDB, Inc.'s licensing terms — but test your specific workload carefully before committing.

Running MongoDB on your own infrastructure — VMs, bare metal, or Kubernetes. This path makes sense at very large scale where cloud pricing becomes a significant cost driver, for compliance requirements that mandate specific data residency or network isolation, or for organizations with strong internal operations teams that already manage large database fleets.

The operational burden is real: you own backups, monitoring, failover testing, security patching, and major version upgrades. Most teams underestimate this cost. Start with Atlas and only consider self-hosting when you have clear, quantified reasons (cost savings, compliance, networking) — not "we like control."

An open-source project that acts as a translation proxy: your application sends MongoDB wire protocol commands to FerretDB, and FerretDB translates them into SQL and stores the data in a Postgres backend. From your application's perspective it looks like MongoDB; under the hood it's Postgres.

Why would you want this? MongoDB, Inc. changed its license in 2018 from AGPL to SSPL — a license that some organizations' legal teams won't accept for hosted services. FerretDB under Apache 2.0 is an escape hatch: keep your existing MongoDB-compatible application code, avoid the SSPL, and store data in Postgres (which most orgs already trust). The trade-off: FerretDB doesn't support 100% of MongoDB's feature set yet, and performance differs from native MongoDB.