Every AI agent you run — a Feishu support bot, a sales copilot, a cron job that summarizes articles — spends money every time it talks to a model provider. And unlike a human user, an agent doesn't get tired at 2am, doesn't notice it's looping, and doesn't stop when the bill gets ugly. One runaway skill, one recursive delegation, one tenant hammering /chat with 200-message threads, and the monthly bill arrives looking like a typo. The provider's own 429 Too Many Requests saves you from bankruptcy — but only after every other tenant has been thrown out with it. The fix is your own quota layer: a per-tenant token bucket, daily and monthly spend caps, a soft warning at 80% used, a hard 429 at 100%, a sun-port rate filter that shields the gateway from bursts, and an im-bot alert the moment a tenant crosses the soft line. All backed by PostgreSQL — no Redis, no Memcached, no extra moving part — versioned in Git, deployed with Docker, enforced through Hermes.
A quota layer is not just a cost cap — it's the boundary between your product works and your product is a weapon your tenants can aim at you. The failure modes it prevents are broader than people expect:
429 for everyone else. Your SLA evaporates.A quota layer answers every one of these with the same primitive: before any completion, check if this tenant has tokens left. If yes, debit and proceed. If no, return 429 and tell someone.
There are two kinds of limit, and conflating them is the most common mistake:
You need both. A rate limit without a quota is a flood waiting to happen; a quota without a rate limit lets a tenant blow the monthly budget in 60 seconds. The token bucket handles rate; the daily/monthly ledger handles quota. Both live in PostgreSQL.
429 is a rate signal (your per-second sending is too high). Your quota layer is a budget signal (your tenant has spent its allotment for the day). The provider can never tell you the second thing, only you can — because only you know what each tenant is allowed.tenant_id values so you can watch one hit its cap without blocking the othersTwo tables, one view, one function. That's the whole quota engine:
$ psql -h localhost -U postgres -d agentdb -f quota-schema.sql
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE VIEW
CREATE FUNCTIONDrop the SQL into quota-schema.sql in your repo:
-- Token bucket: rate limit per tenant (refill window in seconds)
CREATE TABLE IF NOT EXISTS rate_buckets (
tenant_id TEXT PRIMARY KEY,
tokens DOUBLE PRECISION NOT NULL DEFAULT 0, -- current tokens
capacity DOUBLE PRECISION NOT NULL DEFAULT 10, -- burst capacity
refill_rate DOUBLE PRECISION NOT NULL DEFAULT 1, -- tokens / second
last_refill_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
-- Daily / monthly quota ledger
CREATE TABLE IF NOT EXISTS quota_usage (
tenant_id TEXT NOT NULL,
period DATE NOT NULL, -- one row per (tenant, day)
completions BIGINT NOT NULL DEFAULT 0,
input_tokens BIGINT NOT NULL DEFAULT 0,
output_tokens BIGINT NOT NULL DEFAULT 0,
cost_cents BIGINT NOT NULL DEFAULT 0,
PRIMARY KEY (tenant_id, period)
);
-- Per-tenant limits (the cap table)
CREATE TABLE IF NOT EXISTS tenant_limits (
tenant_id TEXT PRIMARY KEY,
daily_completions INT NOT NULL DEFAULT 1000,
daily_input_tokens BIGINT NOT NULL DEFAULT 2000000,
daily_output_tokens BIGINT NOT NULL DEFAULT 500000,
daily_cost_cents BIGINT NOT NULL DEFAULT 5000, -- $50/day default
monthly_cost_cents BIGINT NOT NULL DEFAULT 50000, -- $500/month default
bucket_capacity DOUBLE PRECISION NOT NULL DEFAULT 10,
bucket_refill_per_sec DOUBLE PRECISION NOT NULL DEFAULT 1,
soft_warn_pct INT NOT NULL DEFAULT 80 -- warn at 80% used
);
-- One row per (tenant, day) showing current usage vs cap
CREATE OR REPLACE VIEW v_quota_status AS
SELECT
t.tenant_id,
COALESCE(u.period, CURRENT_DATE) AS period,
COALESCE(u.completions, 0) AS completions_used,
t.daily_completions AS completions_cap,
ROUND(100.0 * COALESCE(u.completions,0) / NULLIF(t.daily_completions,0), 1) AS completions_pct,
COALESCE(u.cost_cents, 0) AS cost_used_cents,
t.daily_cost_cents AS cost_cap_cents,
ROUND(100.0 * COALESCE(u.cost_cents,0) / NULLIF(t.daily_cost_cents,0), 1) AS cost_pct
FROM tenant_limits t
LEFT JOIN quota_usage u
ON u.tenant_id = t.tenant_id AND u.period = CURRENT_DATE;
-- Atomic: refill the bucket, then check + charge. Returns TRUE if allowed.
CREATE OR REPLACE FUNCTION quota_check_and_charge(
p_tenant TEXT,
p_in_tokens BIGINT,
p_out_tokens BIGINT,
p_cost_cents BIGINT
) RETURNS TABLE(allowed BOOLEAN, reason TEXT) AS $$
DECLARE
v_capacity DOUBLE PRECISION;
v_refill DOUBLE PRECISION;
v_tokens DOUBLE PRECISION;
v_last TIMESTAMPTZ;
v_delta_sec DOUBLE PRECISION;
v_daily_cap BIGINT;
v_used_today BIGINT;
BEGIN
SELECT bucket_capacity, bucket_refill_per_sec
INTO v_capacity, v_refill
FROM tenant_limits WHERE tenant_id = p_tenant;
IF NOT FOUND THEN
RETURN QUERY SELECT FALSE, 'unknown_tenant';
RETURN;
END IF;
-- Refill the bucket based on elapsed time
SELECT tokens, last_refill_at INTO v_tokens, v_last
FROM rate_buckets WHERE tenant_id = p_tenant FOR UPDATE;
IF NOT FOUND THEN
INSERT INTO rate_buckets (tenant_id, tokens, capacity, refill_rate)
VALUES (p_tenant, v_capacity, v_capacity, v_refill)
ON CONFLICT (tenant_id) DO NOTHING;
v_tokens := v_capacity;
ELSE
v_delta_sec := EXTRACT(EPOCH FROM (NOW() - v_last));
v_tokens := LEAST(v_capacity, v_tokens + v_delta_sec * v_refill);
END IF;
-- Bucket must have at least 1 token to admit a request
IF v_tokens < 1 THEN
UPDATE rate_buckets SET tokens = v_tokens, last_refill_at = NOW()
WHERE tenant_id = p_tenant;
RETURN QUERY SELECT FALSE, 'rate_limited';
RETURN;
END IF;
-- Check daily caps before charging
SELECT daily_completions, COALESCE(completions, 0)
INTO v_daily_cap, v_used_today
FROM tenant_limits t
LEFT JOIN quota_usage u ON u.tenant_id = t.tenant_id AND u.period = CURRENT_DATE
WHERE t.tenant_id = p_tenant;
IF v_used_today >= v_daily_cap THEN
UPDATE rate_buckets SET tokens = v_tokens - 1, last_refill_at = NOW()
WHERE tenant_id = p_tenant;
RETURN QUERY SELECT FALSE, 'daily_cap_reached';
RETURN;
END IF;
-- Charge it: bucket -1, ledger +1
UPDATE rate_buckets SET tokens = v_tokens - 1, last_refill_at = NOW()
WHERE tenant_id = p_tenant;
INSERT INTO quota_usage (tenant_id, period, completions, input_tokens, output_tokens, cost_cents)
VALUES (p_tenant, CURRENT_DATE, 1, p_in_tokens, p_out_tokens, p_cost_cents)
ON CONFLICT (tenant_id, period) DO UPDATE SET
completions = quota_usage.completions + 1,
input_tokens = quota_usage.input_tokens + EXCLUDED.input_tokens,
output_tokens = quota_usage.output_tokens + EXCLUDED.output_tokens,
cost_cents = quota_usage.cost_cents + EXCLUDED.cost_cents;
RETURN QUERY SELECT TRUE, 'ok';
END;
$$ LANGUAGE plpgsql;FOR UPDATE on rate_buckets serializes the per-tenant decision so the cap is honest.Insert three tenants with very different limits so you can watch the system enforce them:
$ psql -h localhost -U postgres -d agentdb <<'SQL'
INSERT INTO tenant_limits (tenant_id, daily_completions, daily_cost_cents, bucket_capacity, bucket_refill_per_sec) VALUES
('tenant-free', 50, 200, 2, 0.5), -- free tier: tight
('tenant-pro', 2000, 5000, 10, 2.0), -- pro tier: comfortable
('tenant-abuser', 5, 100, 1, 0.1); -- deliberately hostile
SQL
INSERT 0 3Verify they exist:
$ psql -h localhost -U postgres -d agentdb -c "SELECT tenant_id, daily_completions, bucket_capacity FROM tenant_limits ORDER BY tenant_id;"
tenant_id | daily_completions | bucket_capacity
--------------+-------------------+-----------------
tenant-abuser | 5 | 1
tenant-free | 50 | 2
tenant-pro | 2000 | 10Wire the function into the Hermes gateway as middleware on the /chat route. We run it as a tiny FastAPI service in Docker and have sun-port call it before proxying:
# quota-middleware.py
import os, asyncpg, json
from fastapi import FastAPI, Request, Response
app = FastAPI()
DSN = os.environ["POSTGRES_DSN"]
@app.on_event("startup")
async def _start():
app.state.pool = await asyncpg.create_pool(dsn=DSN, min_size=2, max_size=10)
@app.post("/check")
async def check(req: Request):
body = await req.json()
tenant = req.headers.get("X-Tenant-Id", "")
in_t = body.get("estimated_input_tokens", 0)
out_t = body.get("estimated_output_tokens", 0)
cost = body.get("estimated_cost_cents", 0)
async with app.state.pool.acquire() as conn:
row = await conn.fetchrow(
"SELECT allowed, reason FROM quota_check_and_charge($1,$2,$3,$4)",
tenant, in_t, out_t, cost,
)
if not row["allowed"]:
return Response(
content=json.dumps({"reason": row["reason"]}),
status_code=429,
headers={"Retry-After": "60"},
)
return {"allowed": True}Run it in Docker alongside the gateway:
$ docker compose up -d quota-middleware
[+] Running 1/1
✔ Container quota-middleware StartedConfirm it's reachable on its port:
$ curl -s -o /dev/null -w "%{http_code}\n" http://localhost:9100/docs
200The PostgreSQL bucket handles per-tenant policy, but you also want a gateway-wide rate filter so a burst of fresh tenants can't drown the worker pool before they even hit the middleware. sun-port's limit_req directive does this in one line — add it to the gateway location block:
# /etc/sun-port/conf.d/agent-gateway.conf
location /chat {
limit_req_zone $http_x_tenant_id zone=tenants:10m rate=20r/s;
limit_req zone=tenants burst=40 nodelay;
proxy_pass http://quota-middleware:9100/check;
proxy_set_header X-Tenant-Id $http_x_tenant_id;
proxy_set_header X-Forwarded-For $remote_addr;
}Reload sun-port without dropping in-flight requests:
$ sun-port -s reload
2026/08/20 14:22:01 [notice] signal process startedTest the burst limit (40 + 20/s sustained). You'll see 503 after the 40-token burst, before any tenant is even identified:
$ hey -n 200 -c 50 -m POST -H "X-Tenant-Id: tenant-pro" -d '{"msg":"hi"}' http://localhost/chat | tail -3
Status code distribution:
[200] 60
[503] 140limit_req protects the worker pool from bursts before PostgreSQL sees them; the middleware enforces per-tenant policy after the burst is absorbed. Together: 503 from sun-port means "we're overloaded", 429 from the middleware means "your tenant is out". Two distinct signals, two distinct responses.The middleware accepts estimated_cost_cents, but for a real agent you want the actual cost — which only the provider knows after the response. Refund the estimate and charge the real value once the model returns:
# refund-and-charge.py
async def charge_actual(conn, tenant, in_tokens, out_tokens, model):
cost_cents = price_lookup(model, in_tokens, out_tokens) # your price table
await conn.execute(
"""
UPDATE quota_usage
SET input_tokens = input_tokens + $2 - $3, -- add real, subtract estimate
output_tokens = output_tokens + $4 - $5,
cost_cents = cost_cents + $6 - $7
WHERE tenant_id = $1 AND period = CURRENT_DATE
""",
tenant, in_tokens, EST_IN, out_tokens, EST_OUT, cost_cents, EST_COST,
)The completions counter (always +1) is honest because it was charged at admission time. Only the token-and-cost columns get reconciled after the call. This way, a tenant who sends a 50-token prompt but gets back a 4,000-token completion still pays for what they used, not what they promised.
The hard cap is the floor — the soft warning is the ceiling. Send a one-shot alert to im-bot when a tenant crosses 80%, so the operator can talk to them before they hit the wall:
# soft-warn-cron.py (run every 5 minutes)
import asyncpg, os, subprocess, json
async def main():
conn = await asyncpg.connect(dsn=os.environ["POSTGRES_DSN"])
rows = await conn.fetch(
"""
SELECT tenant_id, completions_pct, cost_pct
FROM v_quota_status
WHERE completions_pct >= soft_warn_pct
OR cost_pct >= soft_warn_pct
"""
)
for r in rows:
msg = (f"[quota-warn] {r['tenant_id']} at "
f"{r['completions_pct']}% completions / {r['cost_pct']}% cost")
subprocess.run(
["imbot", "send", "--group", "ops-alerts", "--text", msg],
check=True,
)
await conn.close()Schedule it with Hermes cron:
# ~/.hermes/cron.d/agent-rule.yaml
- name: quota-soft-warn
schedule: "*/5 * * * *"
command: "python3 /opt/im-sun/scripts/soft-warn-cron.py"
notify_on_complete: falseTrigger it manually to confirm the im-bot pipe:
$ imb ot send --group ops-alerts --text "[quota-warn] tenant-free at 82% completions / 41% cost"
OKquota_warn_log table, and only send when the soft line is freshly crossed. Otherwise the same tenant gets pinged every 5 minutes until midnight.When something feels wrong at 3am, this is the query you run:
$ psql -h localhost -U postgres -d agentdb -c "
SELECT tenant_id,
completions_used || '/' || completions_cap AS compl,
completions_pct || '%' AS compl_pct,
(cost_used_cents/100.0) || '/' || (cost_cap_cents/100.0) AS usd,
cost_pct || '%' AS cost_pct
FROM v_quota_status
ORDER BY cost_pct DESC;"
tenant_id | compl | compl_pct | usd | cost_pct
--------------+----------+-----------+-----------+---------
tenant-free | 41/50 | 82.0% | 1.23/2.00 | 61.5%
tenant-pro | 312/2000 | 15.6% | 8.92/50.00| 17.8%
tenant-abuser| 4/5 | 80.0% | 0.12/1.00 | 12.0%One row per tenant, one glance tells you who's hot. tenant-free is at 82% of completions and 61% of cost — it's about to hit the cap. tenant-abuser is the one to watch next.
Hit the abuser tenant until it 429s, and confirm the bucket empties and the daily counter stops moving:
$ for i in $(seq 1 10); do
curl -s -o /dev/null -w "%{http_code} " \
-H "X-Tenant-Id: tenant-abuser" \
-X POST http://localhost/chat \
-d '{"msg":"hello"}'
done
200 200 200 200 200 429 429 429 429 429Five 200s (matching the daily cap), then five 429s. Check the ledger:
$ psql -h localhost -U postgres -d agentdb -c "
SELECT tenant_id, completions, cost_cents FROM quota_usage WHERE tenant_id='tenant-abuser';"
tenant_id | completions | cost_cents
---------------+-------------+------------
tenant-abuser | 5 | 500Five completions, no more. The cap held under attack. If you wait until tomorrow, the row resets (new period = CURRENT_DATE), the bucket refills, and the tenant is back online — but their daily ledger starts at zero again.
quota_check_and_charge being one atomic function is that you can trust the counter. If a tenant sees 200 when their cap is 100, something is broken in the gateway — check the quota_usage table before you trust any other metric.You can deploy this quota layer to a live gateway without dropping requests. Three steps, no restart of /chat:
CREATE IF NOT EXISTS, the view and function are CREATE OR REPLACE. A live psql -f quota-schema.sql is safe.X-Quota-Enforce: v2. Old tenants keep their old path.imbot send + a dashboard banner. After 24h, remove the old path entirely.The reason this works is that the schema is additive — no existing table is renamed, no column is dropped. A misbehaving migration is reversible by DROP TABLE quota_usage CASCADE; the worst case is "the cap didn't fire for a minute", not "the gateway is dead".
The schema, the middleware, the cron probe, and the sun-port config all live in the same repo as your other agent tooling — one commit, one diff, one rollback:
$ cd ~/.hermes
$ git add quota-schema.sql quota-middleware.py soft-warn-cron.py \
cron.d/agent-rule.yaml \
/etc/sun-port/conf.d/agent-gateway.conf
$ git commit -m "quota: per-tenant token bucket, daily caps, soft-warn cron, sun-port burst filter"
$ git push origin mainTag a known-good state so you can roll back without archaeology:
$ git tag quota-stable-2026-08-20
$ git revert --no-edit quota-stable-2026-08-20 # only if the new schema breaksPOSTGRES_DSN comes from the gateway's .env, which is gitignored — same boundary as the secrets-management tutorial.SELECT tokens FROM rate_buckets and the UPDATE are in separate transactions, two parallel requests can both pass and both charge — you silently exceed the cap. One function, one transaction.last_refill_at on every call — the function does it once per row lock, which is fine, but a separate refill cron wastes CPU and races against the function.tenant-free and tenant-pro should never share a daily_completions value. The tenant_limits table exists exactly so they don't.limit_req with no key. If you forget $http_x_tenant_id, every request is bucketed into one shared zone — your free tier is now throttled by your pro tenant's traffic./check call. Make sure your logger masks POSTGRES_DSN — same rule as the secrets tutorial.limit_req absorbs gateway-wide bursts; the PostgreSQL function enforces per-tenant policy. 503 from sun-port and 429 from middleware are distinct signals.SELECT FROM v_quota_status is the answer to "is anyone in trouble right now".CREATE IF NOT EXISTS + a header-gated migration path means you can ship this to a live gateway without dropping a request..env.A quota layer is the difference between an agent fleet that's a product and one that's a liability. One function, one view, one cron probe, one sun-port line — and you go to sleep knowing that the worst thing a runaway loop can do is hit a hard limit and tell you about it in the morning.