39 lines
1.0 KiB
Markdown
39 lines
1.0 KiB
Markdown
# ADR 0001 — Static Backend Configuration
|
|
|
|
**Status**: Draft
|
|
**Date**: 2026-07-23
|
|
|
|
## Context
|
|
|
|
The gateway needs to know which upstream inference servers to route requests to. During initial development, the set of backends is known at deploy time.
|
|
|
|
## Decision
|
|
|
|
Backends are configured via a static TOML config file. The binary reads `config.toml` from the working directory by default, or accepts another path through `--config <PATH>`.
|
|
|
|
```toml
|
|
[[backend]]
|
|
name = "gpu-01"
|
|
address = "http://10.0.1.10:8000"
|
|
|
|
[[backend]]
|
|
name = "gpu-02"
|
|
address = "http://10.0.1.11:8000"
|
|
|
|
[cache]
|
|
block_size = 512
|
|
expiry_seconds = 300
|
|
```
|
|
|
|
`name` is a human-readable identifier for logging/metrics. `address` is the backend base URL. Cache block size and cache-location idle expiry are global startup settings.
|
|
|
|
## Consequences
|
|
|
|
- Simple, no external dependencies.
|
|
- Adding/removing backends requires a config change and restart.
|
|
- Dynamic registration can be added later.
|
|
|
|
## Alternatives Considered
|
|
|
|
- CLI flags per backend, environment variables, dynamic API.
|