At a glance
15
Go services behind one gateway
apps/ in arda-be
93 / 28
Policy routes / marked risk: high
configs/policy.yaml
15
Workloads sharing one HMAC secret
arda-app-secrets in arda-infra
5
Default max concurrent sessions
DefaultMaxConcurrentSessions, reject_oldest
✎ Key Insights in 30 Seconds
Architecture note: An audit of Arda's identity path as the code stands: opaque session cookie in a BFF, Kratos and Hydra at login, stripped and re-injected identity headers, mTLS plus an HMAC workload assertion between services, and what a compromised pod could still do.
Problem Context
Browsers that hold raw JWTs expose them to script, and services that parse client-sent identity headers can be told anything. Arda moves the token behind an opaque cookie and rewrites the identity headers at the gateway, but the interesting question is what the services behind the gateway still have to trust.
Authentication Beyond JWT: How Arda Authenticates Requests, and Where Trust Ends
Architecture & Design
auth-gateway is the browser-facing BFF: it bridges Kratos login to Hydra, stores the session and Hydra token in Redis behind an HttpOnly cookie, matches each request to a risk-tagged route policy, re-validates the IAM user context, and sets identity headers after deleting any the client sent. Backend gRPC requires mTLS and a short-lived HMAC assertion naming source and audience; user context travels as unsigned delegated metadata.
Key Technical Decisions
- 1Opaque cookie plus server-side session: the token never reaches JavaScript, but every request depends on Redis and a revoked permission can live for the 60 s re-check interval on non-high routes.
- 2Route risk levels in YAML: high routes force a fresh IAM lookup and recent auth; unlisted routes return 404, so the file must stay complete.
- 3Hydra subject is iam_users.id, not the Kratos identity id: services get one stable key, at the cost of a mapping table (iam_identity_mappings) to maintain.
- 4mTLS plus an HMAC assertion for gRPC: it removes plaintext and anonymous callers, but one shared secret means any holder can claim any source.
- 5Plain identity headers on the HTTP hop to most services: simple, but safe only while nothing except the gateway can reach them.
Two request paths, and what really sanitises the headers
The earlier version of this article said Traefik runs ForwardAuth on a session cookie and services trust X-User-Id because Traefik sanitises it. Reading the code again, one claim in that was wrong.
There are two request paths into auth-gateway:
GET /auth/checkis the Traefik ForwardAuth endpoint. It reads a Bearer JWT, not a cookie, verifies it, resolves the user in IAM and returns the identity as response headers.arda-infra/traefik/dynamic.ymllists four routers behind it (iam, mdm, platform, workflow).- Everything else reaches the backend through the gateway's own reverse proxy (
BFFHandler.Proxy, mounted on/api/).arda-infra/docs/api-edge-routing.mdsays the other ten services are "reached through auth-gateway's own proxy routes"; production traffic goes fromapi.arda.io.vnthrough Cloudflare and a tunnel toauth-gateway:8080.
In the proxy path the browser holds one opaque cookie (arda_sid, HttpOnly, Secure from config, SameSite=Lax by default). The Hydra access token lives in the server-side session record in Redis; the proxy attaches it as Authorization: Bearer on the upstream call. The part of the session sent to the SPA (session.UserInfo) has no token fields. That is the honest content of "no token in JavaScript": a property of where the token is stored.
What actually protects the headers is the first thing the proxy does with the incoming request: delete every identity header the client sent, then set them again only if a session was found.
// apps/auth-gateway/internal/handler/bff_handler.go
func stripAuthContextHeaders(header http.Header) {
for _, key := range []string{
"X-User-Id", "X-Actor-User-Id", "X-User-Subject",
"X-Tenant-Id", "X-Org-Id", "X-Roles", "X-Permissions",
"X-Global-Roles", "X-Global-Permissions", "X-Global-Admin",
"X-Session-Id", "X-Auth-Version", "X-Auth-Time",
"X-Auth-Risk", "X-Auth-Checked", // ...
} {
header.Del(key)
}
}
Without this, a browser could send X-Tenant-Id: <someone else> and the downstream service would believe it wherever the gateway does not overwrite it. docs/financial-grade-auth-plan.md lists stripping as step 1 and marks it done.
Login bridge and the server-side session
Login splits identity across four systems. docs/kratos-first-identity-design.md says who owns what: Kratos owns credentials, Hydra owns OAuth2/OIDC, IAM owns iam_users, tenants, roles, sessions and audit, and auth-gateway owns the browser session. The canonical user id is iam_users.id; the Kratos identity id sits beside it (iam_users.kratos_identity_id) and must not be used as X-User-Id.
AcceptKratosLogin bridges Kratos and Hydra. After the browser finishes a Kratos flow through /api/kratos/*, it posts the Hydra login_challenge. The gateway does not trust the client's claim of identity: it asks Kratos whoami, maps the identity to an IAM user, and accepts the challenge with the IAM id as subject.
// apps/auth-gateway/internal/handler/bff_handler.go (AcceptKratosLogin, trimmed)
whoami, err := h.kratosWhoami(r, req.KratosSessionToken)
if err != nil { respondError(w, http.StatusUnauthorized, err.Error()); return }
uc, err := h.iamClient.GetUserByKratosIdentityID(r.Context(), whoami.Identity.ID)
if err != nil {
uc, err = h.iamClient.ResolveOrLinkKratosIdentity(r.Context(),
whoami.Identity.ID, whoami.Identity.Traits.Email, whoami.Identity.Traits.Name)
if err != nil { respondError(w, http.StatusBadGateway, "user context unavailable"); return }
}
req.Subject = uc.UserID // Hydra subject is iam_users.id
The plan notes that the legacy direct Hydra accept-login endpoint was removed. TOTP is checked here too, and IAM locks verification for 15 minutes after 5 failures (mfaMaxFailedAttempts, mfaLockDuration in mfa_service.go).
At GET /api/auth/callback the gateway consumes a single-use OAuth state (ConsumeOAuthState), exchanges the code with the server-side PKCE verifier, and creates the session: a JSON blob in Redis (bff:session:<id>, plus a per-user set bff:user_sessions:<userId>). IAM keeps a durable copy in iam_sessions and iam_devices for listing and revoking.
Session limit and what happens at the limit
The default limit is 5 concurrent sessions with OnExceed = "reject_oldest": the sixth login revokes the oldest IAM session (replaced_by_newer). Only "reject_new" errors, mapped to HTTP 429 iam.session.limit_reached.
Two things I noticed. The check is count-then-insert with no lock, so two simultaneous logins can both pass it. And docs/problems/iam.session.limit_reached.md is still an auto-generated skeleton saying status 500 and "an unexpected error occurred", while the handler returns 429; a client following the doc would retry instead of asking the user to close a session.
How stale an authorisation decision may be
A session that is checked once at login is a session that keeps working after the user is disabled or loses a role. docs/financial-grade-auth-plan.md answers with route risk levels, implemented in configs/policy.yaml: 93 route entries, each with auth, risk (public, low, medium, high) and a permission list; 28 of them are risk: high. An unmatched route returns 404 route_not_found instead of reaching the upstream.
For each proxied request the gateway then does, in order: match the route, load the session from Redis, enforce recent auth on high routes, re-validate the IAM context if due, check the route's permissions, and only then set the identity headers.
// apps/auth-gateway/internal/handler/bff_handler.go (Proxy, trimmed)
if !h.recentAuthOK(r, sess) { /* 403 recent_auth_required */ }
forceFreshUser := match != nil && match.Route.Risk == "high"
if !h.ensureSessionUser(r.Context(), sess, forceFreshUser) { /* 401 user_context_unavailable */ }
...
if len(match.Route.Permissions) > 0 && !sess.User.IsGlobalAdmin &&
!permission.HasAny(sess.User.Permissions, match.Route.Permissions...) {
/* 403 insufficient_permissions */
}
Freshness is bounded, not zero. session_auth_check_interval_seconds is 60 in configs/config.yaml, so a revoked permission can keep working on a low or medium route for up to that interval; high routes always re-resolve. IAM's auth_version column is bumped on user, password, role and permission changes and forwarded as X-Auth-Version; disabling a user is caught separately by a Status == "DISABLED" check that deletes the session. The ForwardAuth path has its own in-memory cache keyed by token subject with a 60 second TTL (iam_context_cache_ttl_seconds), skipped on high routes. That cache is per pod and has no invalidation message; the plan lists event-driven invalidation as a goal, and I did not find it implemented.
Bearer tokens on /auth/check go through internal/token/verifier.go; its jwks strategy accepts only RS256 and requires iss, aud, exp and kid, though the compiled-in default in config.go is jwt (shared secret).
Failure mode: IAM is down
ensureSessionUser has a deliberate fail-open branch: if a scheduled re-check cannot reach IAM and the session already has a complete user context, it keeps serving the last validated one, and LastAuthCheck is left untouched so the next request retries. The comment says this avoids logging everyone out during an outage. high routes return 401 instead. The cost: a disabled user stays active on non-high routes for the length of an IAM outage.
Step-up is weaker than its name for users without MFA
POST /api/auth/step-up refreshes AuthTime, which recentAuthOK compares against recent_auth_window_seconds (300). If the user has TOTP enrolled, a valid code is required. If not, the request body {"confirm": true} is enough. Nothing re-checks the password. docs/refactor-program/03-identity-and-security.md section 9 calls a generic confirmation boolean "not sufficient" for high-risk actions; that branch still exists.
Workload identity: mTLS plus a signed assertion
If any pod can call any other pod and say "I am workflow-service", the edge checks are decoration. docs/refactor-program/03-identity-and-security.md states the invariant ("internal network location is not service identity") and 09-mtls-runbook.md describes two required layers.
Layer 1, mTLS. libs/go/arda-grpc/identity/tls.go builds TLS 1.3 credentials from ARDA_GRPC_CA_FILE, ARDA_GRPC_CERT_FILE and ARDA_GRPC_KEY_FILE. Servers use tls.RequireAndVerifyClientCert; missing files fail startup, with no plaintext fallback. The runbook asks for one certificate per workload from one private CA.
Layer 2, a signed assertion in gRPC metadata key x-service-auth. identity.Issue builds v1.<base64url(json)>.<base64url(HMAC-SHA256)>:
// libs/go/arda-grpc/identity/identity.go
type claims struct {
Version string `json:"v"`
Source string `json:"src"`
Audience string `json:"aud"`
IssuedAt int64 `json:"iat"`
ExpiresAt int64 `json:"exp"`
Nonce string `json:"nonce"`
}
// Verify: constant-time signature compare, then
// c.Audience == expectedAudience
// now >= iat - 30s and now < exp
The client interceptor issues a one-minute token (the library caps TTL at 5 minutes) for one audience; the server interceptor verifies it and checks claims.Source against an allowlist, per method if needed. For example, loan-service accepts workflow-service and finance-service, and only platform-service may call IAM's ListActiveTenants.
The assertion carries source, audience and time. It does not carry the user. The package comment says so: "User/session metadata is separate and remains delegated context." On the plain HTTP hop, the gateway signs calls to IAM's internal routes (IAM requires source == "auth-gateway") and to ai-service for /api/ai/ and /api/rag/. The other proxied upstreams (crm, finance, hrm and so on) receive user headers with no signature.
Replay
SignRequest documents the limit itself: the token "carries a nonce for audit/correlation, but there is no server-side one-time-token store, so the window is reduced, not eliminated". A captured assertion is replayable to its audience until it expires.
What a compromised pod can and cannot do
This is what the old copy got wrong. Assume an attacker has code execution in one backend pod, say crm-service.
| Attacker holds | Can do | Cannot do (per code) |
|---|---|---|
The pod's own cert, key, and ARDA_SERVICE_AUTH_SECRET | Complete mTLS handshakes to any gRPC server; mint a valid assertion with any src and any aud | Skip mTLS (no plaintext path, TLS 1.3 required) or obtain a cert for another name without the CA key |
Network reach to an HTTP service that checks only X-Auth-Checked | Send X-Auth-Checked: true with any tenant, user, X-Global-Admin | Nothing in code |
| Nothing but the browser | Send forged identity headers | Get them past the gateway: stripAuthContextHeaders deletes them before the proxy sets its own |
The reasons, each from code:
- One secret for everyone.
arda-infra/k8s/apps/*.yamlhas 15 workloads readingARDA_SERVICE_AUTH_SECRETfrom the same Secretarda-app-secrets. The assertion'ssrcis a string inside the signed payload, so whoever has the secret can sign as any service. The allowlists only stop callers without the secret. - The certificate is not bound to the claimed source. The server requires a certificate chaining to the CA, but
tls.gonever looks at its subject and the interceptor readssrcfrom the HMAC token only. mTLS says "one of ours"; the assertion says "which one" on its own word, as09-mtls-runbook.mddescribes. - User context is delegated, unsigned metadata. Over gRPC,
x-user-id,x-tenant-id,x-rolesandx-permissionsare copied between contexts (UnaryServerMetadataPropagate), and a caller that passed the service check can set any of them. IAM'sWithTenantand the HRM and workflow routers accept a request whenX-Auth-Checked == "true"and a tenant header is present. - Some services do no authorisation of their own, by decision.
platform-service/internal/handler/authz.gosays: "platform-service performs no authn/authz of its own and must not add a parallel policy store", relying on the gateway to strip and rewrite headers, soX-Auth-CheckedandX-Global-Admin"can only originate from a verified gateway session". That holds only if nothing but the gateway can reach the service. I found noNetworkPolicymanifest inarda-infra; the refactor plan wants network policy but "does not rely on network policy alone".
So one compromised pod can reach anything exposed over internal HTTP as any user and tenant, and any gRPC method whose allowlist it can name itself into. That is not "zero trust" in the sense the old title promised. It is mutual TLS between workloads plus an edge that is hard to bypass from the internet. What the design does buy: a stolen browser cookie yields one user's session, bounded by role and tenant membership, and a forged header from outside never reaches a service.
Is it deployed?
docs/refactor-program/runtime-evidence-20260825.md records that on 2026-08-25 the live cluster had no arda-grpc-tls-* Secrets and no ARDA_GRPC_* mounts. The current arda-infra manifests reference those variables for 12 services, but I have no later runtime evidence: I am describing code and manifests, not the cluster.
! careful
The HMAC assertion proves "a holder of the shared secret", not "this specific service", and it does not cover the user headers.
The plan against the code, and what I'd check next
docs/financial-grade-auth-plan.md (last updated 2026-06-30) has a seven-step implementation order and a status list. Here is each item against what I found in the code.
| Plan item | Status in code |
|---|---|
| 1-2. Strip client headers; route risk metadata | Implemented |
| 3. JWKS local verification as production default | Implemented as a strategy; infra sets TOKEN_STRATEGY=jwks; code default is still jwt |
| 4. Short-lived IAM context cache | Implemented (60 s, skipped on high); per pod, no invalidation events |
5. auth_version and invalidation | Column and header implemented; the plan's own "Next" says the cache does not yet compare versions against token/session claims |
| 6. Recent-auth and MFA step-up | Recent-auth enforced on high routes; step-up without MFA is a confirm flag |
| 7. Audit events and dashboards | Not verified; iam_audit_logs exists and the session handler writes session_created, but I did not check coverage |
One gap the repo's own docs name and I could not find closed:
- CSRF.
03-identity-and-security.mdsection 8 requires "CSRF/origin checks cover all cookie-authenticated state-changing endpoints". I found no CSRF token orOrigincheck in auth-gateway; the Kratoscsrf_tokeninarda-mfe/packages/authbelongs to Kratos flows. What the gateway does have is an exact-match CORS allowlist andSameSite=Lax. Both sites share the registrable domainarda.io.vn, soSameSitedoes not separate them, and a CORS allowlist limits who can read responses, not who can send a simple cross-origin POST. I would add an Origin check on mutating proxy routes.
What I would check next
- Whether the cluster has a
NetworkPolicythe repo does not show; if not, default-deny plus an allow rule fromauth-gateway. - Bind the assertion's
srcto the client certificate's SAN in the server interceptor. - Sign the user context, for example a short-lived JWT minted by the gateway per request.
- Replace the
confirmbranch of step-up with real re-authentication.
I would not copy this header-injection design to a system where internal pods are less trusted than I assume these are.
Verified repository evidence
auth · direct
Layered auth boundary: auth-gateway BFF with Redis-backed sessions behind an arda_sid cookie, Ory Kratos/Hydra for login, and mTLS plus short-lived signed workload assertions between services.
↳ Gateway strips client-supplied identity headers and sets them from the validated session; it signs a short-lived service token for internal calls.
arda-labs/arda-be ↗
Key Engineering Takeaways
- "No token in JavaScript" is true for the cookie flow; "zero trust" is not true of the internal network today.
- The workload assertion authenticates a service, never a user, and has no replay store.
- Most of the plan's checklist is implemented; CSRF checks, signed user context and per-workload secrets are the visible gaps.