kvca_proxy/docs/adr/0001-static-backend-config.md

ADR 0001 — Static Backend Configuration

Status: Draft
Date: 2026-07-23

Context

The gateway needs to know which upstream inference servers to route requests to. During initial development, the set of backends is known at deploy time.

Decision

Backends are configured via a static TOML config file. The binary reads config.toml from the working directory by default, or accepts another path through --config <PATH>.

[[backend]]
name = "gpu-01"
address = "http://10.0.1.10:8000"

[[backend]]
name = "gpu-02"
address = "http://10.0.1.11:8000"

[cache]
block_size = 512
expiry_seconds = 300

name is a human-readable identifier for logging/metrics. address is the backend base URL. Cache block size and cache-location idle expiry are global startup settings.

Consequences

• Simple, no external dependencies.
• Adding/removing backends requires a config change and restart.
• Dynamic registration can be added later.

Alternatives Considered

• CLI flags per backend, environment variables, dynamic API.