Sparse Index

A sparse index is a DynamoDB secondary index that contains only the items which actually have the index’s key attributes set. If an item doesn’t have the attribute used as the index partition key, DynamoDB never writes that item into the index. The index stays small, and querying it is cheap regardless of how large the base table grows.

This isn’t a feature you enable. It’s a consequence of how DynamoDB populates indexes, which you exploit deliberately.

How it works

When you create a Global Secondary Index, you nominate an attribute as the index partition key and optionally another as the index sort key. On every write to the base table, DynamoDB checks whether the item has those attributes:

• Attribute present → the item is projected into the index
• Attribute absent → the item is silently skipped

There’s no error and no warning. The item simply isn’t in the index. That absence is the whole mechanism.

The practical consequence: a table with 50 million items can have a GSI containing 200 items, if only 200 items carry the index key attribute. Queries against that index scan 200 items’ worth of index space, not 50 million.

The canonical example: a work queue

Suppose you have an orders table and an operations dashboard that needs “show me every order awaiting manual review.” Orders in that state are a tiny fraction of total orders, and they leave the state within hours.

The naive approach is a GSI partitioned on status, then querying STATUS#pending_review. That works, but every order in every status is now duplicated into the index. You pay write capacity for all of them and storage for all of them, to serve a query about a handful.

The sparse approach writes the index key only when the condition holds:

PK: ORDER#01HVMK3P2Q     SK: #METADATA
    status: pending_review
    gsi1pk: REVIEW_QUEUE          ← only present while under review
    gsi1sk: 2026-08-05T14:22:00Z

When the order clears review, you REMOVE gsi1pk and gsi1sk in the update expression. DynamoDB deletes the item from the index. The queue self-cleans.

Query(GSI1, gsi1pk = REVIEW_QUEUE)

One query returns exactly the orders under review, oldest first, with no filter expression and no wasted reads.

Why this beats a filter expression

The alternative most people reach for is a FilterExpression. It produces the same result set and is dramatically more expensive.

DynamoDB applies filters after reading the items. You are charged read capacity for every item examined, not every item returned. Querying a status GSI containing 50 million orders and filtering down to 200 costs you the read capacity of 50 million items.

	Filter on a dense index	Sparse index
Items read	All items in the partition	Only matching items
RCU charged	Proportional to table size	Proportional to result size
Write amplification	Every write hits the index	Only qualifying writes hit the index
Storage cost	Full projection of the table	Full projection of the subset
Query latency	Grows with table size	Flat

The filter approach degrades as the table grows. The sparse approach doesn’t.

Four patterns that use sparse indexes

Work queues and pending states. Anything with a transient status: unprocessed uploads, orders awaiting fulfilment, flagged content, failed jobs pending retry. The attribute exists while the work is outstanding and is removed on completion.

Soft deletes and archives. Set archivedAt only on archived records and index on it. Your “recently archived” view is a sparse query. The active-record queries never touch it.

Entity-type enumeration. In a single-table design, “list every tenant” is awkward because tenants share a table with users, projects, and subscriptions. Give only the tenant metadata record a gsi1pk: TENANT_LIST attribute. The index contains one item per tenant and nothing else. This is the pattern used in the SaaS multi-tenant schema.

Exception tracking. Records that need attention: subscriptions past due, accounts over quota, schemas failing validation. Write the index attribute when the exception condition becomes true, remove it when it resolves.

The common shape across all four: a boolean-ish condition that is true for a small minority of items, and where you want to enumerate the minority.

ElectroDB implementation

ElectroDB handles sparse indexes through optional attributes. If a composite attribute is undefined, the index key isn’t written:

export const OrderEntity = new Entity({
  model: { entity: "order", version: "1", service: "shop" },
  attributes: {
    orderId: { type: "string", required: true },
    status: { type: "string", required: true },
    // Optional. Present only while the order is in review.
    reviewQueuedAt: { type: "string", required: false },
  },
  indexes: {
    primary: {
      pk: { field: "pk", composite: ["orderId"], template: "ORDER#${orderId}" },
      sk: { field: "sk", composite: [], template: "#METADATA" },
    },
    reviewQueue: {
      index: "gsi1",
      pk: { field: "gsi1pk", composite: [], template: "REVIEW_QUEUE" },
      sk: { field: "gsi1sk", composite: ["reviewQueuedAt"] },
    },
  },
}, { client, table });

Setting reviewQueuedAt puts the order in the queue. Removing it takes the order out.

Common mistakes

Forgetting that removal is a write. Taking an item out of a sparse index costs a write unit on the base table plus a delete on the index. It’s cheap, but it isn’t free, and it has to actually happen. Code paths that mark work complete without removing the index attribute leave orphans in the queue forever.

Using a static partition key at high write volume. gsi1pk: REVIEW_QUEUE puts every queued item on one partition. Fine for a queue of hundreds. If your queue depth reaches thousands of writes per second, you need write sharding on the index key.

Setting the attribute to an empty string instead of removing it. An empty string is a present value in older SDK behaviour and null is not a valid key type. Use REMOVE in the update expression, not SET x = "".

Over-projecting. Sparse indexes are usually read for identification, then the full item is fetched by primary key. KEYS_ONLY or a narrow INCLUDE projection is often correct and cuts storage cost meaningfully.

Assuming immediate consistency. GSIs are eventually consistent. An item removed from a sparse index may briefly still appear in query results. Design consumers to tolerate re-reading an item that has already been processed.

Related terms

• GSI Overloading — sharing one index across multiple access patterns
• Partition Key — the attribute whose presence determines index membership
• Single-Table Design — where sparse indexes do the most work

---

Sparse indexes are easy to reason about in isolation and easy to lose track of across a dozen entities. I’m building singletable.dev to show which items land in which index, visually, before you deploy.