Verified: 2026-10-17By Alex Chen (Lead BGP Architect)Redis & High Concurrency9 Min Read

Caching Strategies for High-Volume IP Lookups: Redis and TTLs

Q: What is the optimal Redis TTL for caching IP geolocation lookups?

The optimal TTL depends on the connection type: Datacenter and cloud server subnets (AWS, DigitalOcean) should be cached for 24 to 72 hours because server ASN ownership is stable. Fixed residential broadband IPs should be cached for 6 to 12 hours. Mobile cellular CGNAT IPs should use shorter TTLs of 1 to 2 hours due to frequent carrier dynamic pool rotation.

Q: Why should IPv6 addresses be normalized before being used as cache keys?

IPv6 addresses can be written in multiple equivalent formats (e.g., 2001:db8::1 vs 2001:0db8:0000:0000:0000:0000:0000:0001). If an application caches raw user input strings without canonicalizing them, the same device will produce multiple distinct cache keys, degrading cache hit ratios and wasting Redis memory.

Q: What is "Negative Caching" and why is it critical for IP lookups?

Negative caching stores failed lookups (e.g., rate limits, timeouts, or Bogon non-routable IPs) with a short TTL (such as 60 seconds). If a malicious scraper blasts your system with millions of fake or internal IP addresses, negative caching stops your workers from continuously hitting downstream APIs or databases, preventing cache penetration attacks.

Q: How does a two-tier (L1 In-Memory + L2 Redis) cache architecture work?

Level 1 is a fast in-process memory LRU cache (e.g., quick-lru in Node.js) stored directly in application heap RAM, resolving in < 5 microseconds. Level 2 is a shared Redis cluster resolving in 1 to 2 milliseconds over the local LAN. L1 absorbs immediate request spikes, while L2 shares cached results across all application server pods.

A high-concurrency systems design guide for caching IP intelligence, exploring two-tier memory hierarchies, connection-aware TTL sizing, and Redis memory optimization.

Quick Answer

To handle tens of millions of daily IP lookups without budget exhaustion, deploy a two-tier caching hierarchy: Level 1 in-process LRU cache (microsecond resolution) backed by Level 2 shared Redis (1ms LAN latency). Apply adaptive TTLs based on IP type: 24 to 72 hours for static datacenter subnets, 6 to 12 hours for residential broadband, and 1 to 2 hours for mobile CGNAT pools.

Two-Tier Hierarchy: L1 in-memory LRU cache absorbs local pod spikes; L2 Redis shares state across clusters.
Adaptive TTL Policies: Size cache duration according to subnet volatility (Datacenter vs Mobile).
IPv6 Key Normalization: Canonicalize hex formats to prevent duplicate cache entries and RAM bloat.

1. The Two-Tier Caching Architecture

High-throughput distributed systems decouple lookup latency by combining local heap RAM with centralized caching:

Level 1: In-Process LRU (RAM Heap):Capacity: 50,000 entries | Latency: 0.005 ms | Hit Rate: ~40-60%

Level 2: Distributed Redis Cluster:Capacity: 5,000,000 entries | Latency: 1.2 ms | Hit Rate: ~85-95%

Level 3: Authoritative Origin (MMDB / API):Triggered only on double-miss (< 5% of all traffic).

2. Adaptive TTL Sizing by Connection Type

Applying a blanket 1-hour TTL wastes money on static subnets, while a 30-day TTL serves stale data to mobile devices. Use adaptive rules:

Adaptive TTL Optimization Matrix for IP Intelligence Caching
Subnet Classification	Recommended Cache TTL	Real-World Reason	Eviction Risk
Datacenter / Cloud Server (AWS, DO)	24 to 72 Hours	Server IPs are statically bound to VMs for months	Extremely low (Ownership rarely changes)
Fixed Residential Broadband (Fiber/Cable)	6 to 12 Hours	DHCP leases typically last 24 to 168 hours	Low (Subnets remain regionally stable)
Mobile Cellular (4G LTE / 5G CGNAT)	1 to 2 Hours	Phones rotate dynamically across carrier pools	Moderate (IP reassigned to new user)
Bogon / RFC 1918 Private Ranges	7 Days	Private ranges (10.0.0.0/8) are permanently non-routable	Zero risk
Negative Cache (API Failure / Timeout)	60 Seconds	Protects origin during outages while allowing fast recovery	Temporary shield

Multi-Tier IP Cache Cascade Decision Tree


                 [ Incoming IP Address Lookup Request ]
                                   |
           +-----------------------+-----------------------+
           |                                               |
   [ Level 1: In-Process LRU ]                             |
   Lookup directly in heap RAM                             |
   Resolved in < 5 microseconds?                           |
           |                                               |
     +-----+-----+                                         |
     |           |                                         |
  [ Hit ]     [ Miss ]                                     |
  Return      Cascade to Level 2                           |
                 |                                         |
           [ Level 2: Shared Redis Cluster ]               |
           Query normalized key: "geoip:1.1.1.1"           |
           Resolved in 1 - 2 milliseconds?                 |
                 |                                         |
           +-----+-----+                                   |
           |           |                                   |
        [ Hit ]     [ Miss ]                               |
        Populate L1 Cascade to Level 3                     |
        Return         |                                   |
                 [ Level 3: MMDB File / Cloud API ] <------+
                 Fetch authoritative record
                 Set Redis TTL (6h - 24h) based on ASN type
                 Populate L1 & L2 caches; return result

3. Production Implementation: Node.js with ioredis

A production implementation enforcing two-tier lookups and canonical IPv6 key hashing:

Canonicalize Cache Key:const cacheKey = 'ip:' + (ip.includes(':') ? normalizeIPv6(ip) : ip);

Redis Set with Adaptive TTL:await redis.set(cacheKey, JSON.stringify(data), 'EX', getAdaptiveTTL(data.asn_type));

4. Practical Diagnostics: Inspecting Redis Memory and Hit Ratios

Monitor your Redis cluster's memory utilization and cache efficiency via redis-cli:

Inspect Cache Hit Ratio:redis-cli info stats | grep -E "keyspace_hits|keyspace_misses"

Check Memory Footprint:redis-cli info memory | grep "used_memory_human"

Frequently Asked Questions

What is the optimal Redis TTL for caching IP geolocation lookups?