Indexing Best Practices

A condensed summary of the 25 most important indexing practices drawn from every page in this section.

Do not use RPC as your database: Persist events in Postgres/warehouse (Indexing Basics).
Idempotent primary keys: (signature, ix_index) or (pubkey, write_version) prevent duplicate rows on webhook retry.
Store ingestion cursor: Last processed slot per source for crash recovery (Building an Indexer).
Parse instruction bytes, not logs: Logs are auxiliary; discriminators in data are canonical (Parsing Program Data).
Walk inner instructions: CPI-heavy programs hide logic in meta.innerInstructions.
Version IDL per deploy: Pin IDL artifact to program release tag in parser CI.
Unknown instruction bucket: Store hex payload for forward-compatible upgrades.
Commitment policy explicit: Index confirmed for UX; promote to finalized for finance if needed.
Gap detection alerts: Monitor slot monotonicity on Yellowstone/webhook streams.
RPC backfill reconciliation: Nightly job compares stream tip vs DB max slot (Yellowstone gRPC).
Verify webhook HMAC: Reject unsigned POSTs (Webhooks).
ACK fast, process async: Return 200 in <1s; queue heavy parsing.
Filter early at source: Geyser/Yellowstone owner filters save egress (Geyser Plugins).
Avoid mainnet GPA in indexer backfill: Use getBlock archives or provider exports with rate limits.
Schema migrations planned: Add columns nullable; dual-write during program upgrades.
Decimal tables for tokens: Join mint metadata; never assume 9 decimals.
Read replica for BI: Dashboards never hit primary writer (Analytics & Dashboards).
Document metric definitions: One SQL source of truth for DAU/volume/TVL.
Price oracles at block time: USD metrics need historical price, not spot guess.
Retention policy: Archive cold partitions; chain data grows without bound.
Test parsers with golden txs: Mainnet fixture base64 in repo for regression.
Scale consumers horizontally: Kafka/queue between ingest and writers.
Secure gRPC tokens: Rotate Yellowstone credentials; audit access.
Do not expose raw chain tables publicly: Curate API with auth and pagination.
Plan provider exit: Abstract ingest interface to swap Yellowstone vs webhook vs self-host.

FAQs

When is RPC backfill enough?

Never for product analytics at scale - only for one-time migrations or tiny address sets.

Webhook vs Yellowstone?

Webhooks for moderate event volume and simplicity; Yellowstone for firehose programs.

How does indexing relate to DAS?

DAS serves asset reads; indexing captures custom program business events DAS does not model.

Indexing Basics - section entry
Building an Indexer - implementation guide
RPC Best Practices - RPC limits
Optimization Best Practices - tx landing

Stack versions: This page was written for Agave 4.1.1, Solana CLI 3.0.10, Anchor 0.32.1, anchor-lang 0.32.1, Rust 1.91.1, @solana/kit 7.0.0, Surfpool 0.12.0, and LiteSVM 0.6.x.

Indexing Best Practices

A condensed summary of the 25 most important indexing practices drawn from every page in this section.

Do not use RPC as your database: Persist events in Postgres/warehouse (Indexing Basics).

Idempotent primary keys: (signature, ix_index) or (pubkey, write_version) prevent duplicate rows on webhook retry.

Store ingestion cursor: Last processed slot per source for crash recovery (Building an Indexer).

Parse instruction bytes, not logs: Logs are auxiliary; discriminators in data are canonical (Parsing Program Data).

Walk inner instructions: CPI-heavy programs hide logic in meta.innerInstructions.

Version IDL per deploy: Pin IDL artifact to program release tag in parser CI.

Unknown instruction bucket: Store hex payload for forward-compatible upgrades.

Commitment policy explicit: Index confirmed for UX; promote to finalized for finance if needed.

Gap detection alerts: Monitor slot monotonicity on Yellowstone/webhook streams.

RPC backfill reconciliation: Nightly job compares stream tip vs DB max slot (Yellowstone gRPC).

Verify webhook HMAC: Reject unsigned POSTs (Webhooks).

ACK fast, process async: Return 200 in <1s; queue heavy parsing.

Filter early at source: Geyser/Yellowstone owner filters save egress (Geyser Plugins).

Avoid mainnet GPA in indexer backfill: Use getBlock archives or provider exports with rate limits.

Schema migrations planned: Add columns nullable; dual-write during program upgrades.

Decimal tables for tokens: Join mint metadata; never assume 9 decimals.

Read replica for BI: Dashboards never hit primary writer (Analytics & Dashboards).

Document metric definitions: One SQL source of truth for DAU/volume/TVL.

Price oracles at block time: USD metrics need historical price, not spot guess.

Retention policy: Archive cold partitions; chain data grows without bound.

Test parsers with golden txs: Mainnet fixture base64 in repo for regression.

Scale consumers horizontally: Kafka/queue between ingest and writers.

Secure gRPC tokens: Rotate Yellowstone credentials; audit access.

Do not expose raw chain tables publicly: Curate API with auth and pagination.

Plan provider exit: Abstract ingest interface to swap Yellowstone vs webhook vs self-host.

FAQs

When is RPC backfill enough?

Never for product analytics at scale - only for one-time migrations or tiny address sets.

Webhook vs Yellowstone?

Webhooks for moderate event volume and simplicity; Yellowstone for firehose programs.

How does indexing relate to DAS?

DAS serves asset reads; indexing captures custom program business events DAS does not model.

Stack versions: This page was written for Agave 4.1.1, Solana CLI 3.0.10, Anchor 0.32.1, anchor-lang 0.32.1, Rust 1.91.1, @solana/kit 7.0.0, Surfpool 0.12.0, and LiteSVM 0.6.x.

Indexing Best Practices

FAQs

Related

Search Documentation

Indexing Best Practices

FAQs

Related