CacheForge
A concurrent in-memory cache server with TCP networking, least-recently-used eviction, and expiring keys.
Built to explore the relationship between correct shared state, asynchronous I/O, and measured performance.
operations / second
Median of three trials at 16 clientsmeasured operations
Every response checked across nine trialsserver worker threads
Asynchronous networking; synchronized cache accessMeasured throughput
Median operations per second across three trials.
Latency alongside throughput
| Clients | Ops/s | p95 µs | p99 µs |
|---|---|---|---|
| 1 | 16,435 | 74.6 | 105.1 |
| 4 | 64,167 | 80.9 | 103.1 |
| 16 | 69,512 | 285.2 | 347.8 |
Latency columns show the median of each trial's percentile. Higher concurrency increased throughput and request latency.
Benchmark methodology
80% GET and 20% SET; 64-byte values; one key per client; one outstanding request per connection. Each client performs 100 warmup operations followed by 2,000 measured operations. Cache capacity: 1,024 entries.
Darwin 25.6.0 · arm64 · 14 logical CPUs · Python 3.12.14
- Client and server share the same computer.
- Python client overhead is included.
- Closed-loop workload; not a saturation or overload test.
- Small hot-key workload; no eviction or expiration pressure.
- Latency summary is the median of trial percentiles.
Request path
Local clients send newline-delimited commands.
Bounded input, a 30-second operation timeout, and serialized callbacks per connection.
A mutex protects a hash map and LRU list. A monotonic clock determines expiration.
Asynchronous writes return values, status, or errors.
Example session
Protocol illustration. The server runs locally; this page is a project showcase.
SET greeting 0 hello world OK GET greeting VALUE hello world DELETE greeting DELETED GET greeting NOT_FOUND
SET accepts a key, TTL in milliseconds, and a printable ASCII value. A TTL of zero disables expiration.
Correctness checks
- Six core test cases: storage, LRU, expiration, overwrites, validation, and concurrency.
- Deterministic expiration tests using an injected clock.
- Network checks for split requests, pipelining, invalid commands, and oversized input.
- Sixteen concurrent network clients with response verification.
Engineering tradeoffs
- A single cache mutex simplifies correctness but limits parallel cache access.
- Capacity is bounded by entry count, not an exact memory budget.
- Expiration cleanup scans the cache when full and during STATS; these operations can take linear time.
- In-memory only: no persistence, replication, authentication, or TLS. The server binds to localhost.