Blockchains are slow at queries. An RPC call to retrieve all transactions for an address requires iterating through the entire chain. A query for transactions in a date range has no index; you must scan everything. This is operationally unacceptable for applications that need fast responses. A caching layer solves this: extract blockchain data into a queryable database, index it, and serve requests from cache instead of hitting the blockchain.
Why Caching Matters for Web3
Blockchain Queries Are Expensive
Blockchains are optimized for consensus and immutability, not query performance. Ethereum stores state in a Merkle tree; finding a specific value requires traversing the tree. A single query can consume significant bandwidth and computation. For end-user applications, this is too slow.
RPC Endpoints Have Rate Limits
Public RPC endpoints (Infura, Alchemy) rate-limit requests to prevent abuse. If your application hits those limits, queries fail. Caching reduces the number of blockchain queries by answering requests from local data.
Cost
Running your own RPC node requires hardware and bandwidth. Public RPC endpoints charge per request or per month. A caching layer amortizes these costs across multiple queries.
Architecture
Blockchain Follower (Indexer)
A service listens to new blocks on the blockchain. For each block, it extracts relevant data (transactions, events, state changes) and writes it to a database. This service must keep pace with block production (roughly one block per 12 seconds on Ethereum). It should also handle chain reorganizations (reorgs) when the blockchain reorders blocks due to consensus rules.
Database (PostgreSQL or Similar)
Extracted data is stored in a relational database with proper schema. Indexes are created on frequently queried fields (address, token, date). Queries run against this database are orders of magnitude faster than querying the blockchain directly.
Schema design is critical. A naive design (one row per transaction) becomes unwieldy; a transaction with 100 events requires 100 rows. Better designs normalize the schema, use JSON columns for variable data, and index strategically.
API Layer
Applications query the cache via REST or GraphQL APIs. The API layer enforces access control (who can query what), rate limits (how many queries per user), and logging.
Real-Time Updates
As new blocks arrive, the cache is updated. Applications using WebSockets or Server-Sent Events can receive live updates. A user viewing a dashboard sees balances update in real time without page refreshes.
Data Flow Example
You want to display all transactions for an address. Traditional approach: call eth_getTransactionByHash for each transaction in the chain until you find matches. This is slow. With caching: the indexer stores all transactions in PostgreSQL. A query like SELECT * FROM transactions WHERE from_address = '0xABC...' returns instantly, and results are cached in Redis for even faster access next time.
Design Considerations
Freshness vs. Performance Tradeoff
If you require sub-second freshness (cache is updated as blocks arrive), you need fast indexing and low-latency database updates. If you can tolerate 10-30 second lag (cache updates periodically), infrastructure is simpler and cheaper.
What to Cache
Cache high-query-volume data: transaction history, token balances, event logs. Do not cache everything; selective caching reduces storage and keeps indexes efficient.
Handling Chain Reorgs
Blockchains reorg occasionally. The canonical chain can change, making previously final blocks non-final. Your indexer must detect reorgs and backfill data. This adds complexity; production systems use libraries (The Graph, Ponder) that handle this automatically.
Storage Growth
Blockchain data grows continuously. A full Ethereum archive is multi-terabytes. A cache of recent data (last 6 months) may be terabytes. Storage growth must be managed; archival of old data and data compression help.
Tools and Frameworks
The Graph
A decentralized indexing service. You define a schema and write subgraphs. The Graph indexes your data and hosts it. Applications query The Graph API. This outsources infrastructure; the tradeoff is less control and potential downtime if The Graph experiences issues.
Custom Indexers (go-ethereum, ethers.js)
Write your own indexer using client libraries. You have full control but must handle reorgs, performance tuning, and uptime yourself. This is preferred for applications where uptime and performance are critical.
Database
PostgreSQL is standard. Alternatives include DuckDB (analytical queries), ClickHouse (time-series data), and MongoDB (flexible schema). Choice depends on query patterns and data shape.
Caching (Redis, Memcached)
Store frequently accessed queries in Redis. When a request comes in, check Redis first. If found, return cached result. If not, query PostgreSQL and cache the result. This adds latency only on cache misses.
Smart Contract Queries and ABI Decoding
Smart contracts emit events. An indexer must decode events to make them queryable. This requires the contract's ABI (Application Binary Interface). An ERC20 Transfer event, when decoded, shows from, to, and value clearly. Raw logs are hex-encoded; decoding is necessary for usability.
When to Build vs. Buy
Build a custom indexer if: (1) you need real-time updates (sub-second), (2) your queries are custom or proprietary, or (3) you run mission-critical applications where uptime cannot be outsourced. Use The Graph if: (1) your indexing needs are standard, (2) you want to outsource operations, or (3) your budget is tight and you can tolerate The Graph's uptime SLA.
Conclusion
Blockchain applications require a caching layer to be usable. Blockchains are not databases; they are ledgers optimized for consensus and immutability. A cache extracts relevant data, indexes it, and makes it queryable. This is a solved problem; established tools (The Graph, custom indexers) handle this well. The design choice is whether to build or use a managed service.
