At a glance
15
Go services in one workspace
Counted from go.work and apps/
18
Shared dirs in libs/go
17 hand-written plus generated arda-proto
13
Proto domain directories
Docs still say 11
13
Micro-frontend remotes
From federation.routes.ts, plus the shell
✎ Key Insights in 30 Seconds
Architecture note: An engineer's map of Arda as the repo stands today: 15 Go services of very different sizes, gRPC inside and HTTP at the edge, NATS events, 18 shared libraries and their coupling cost, the newer loan/deposit/capital/statistical/AI services, the GHCR to Argo CD path, and a candid list of what is still rough.
Problem Context
A back office split into 15 Go services needs one place that says what each service owns, how they call each other and what they share. Otherwise the boundaries live in people's heads and the diagrams drift: the previous copy of this article said 9 services, and the repo now has 15.
How Arda Is Put Together: 15 Go Services, One Workspace
Architecture & Design
Browser traffic enters through Cloudflare and a Tunnel to `auth-gateway`, which routes to 14 domain services in one Go workspace. Services call each other over gRPC (mTLS, signed workload assertions), publish events to NATS JetStream from per-service outbox tables, and reach Zeebe only through `workflow-service`. Each service owns its schema and migrates it at startup on a CloudNativePG cluster; images go to GHCR and Argo CD Image Updater rolls digests into `arda-infra`.
Key Technical Decisions
- 1One Go workspace with 15 modules: atomic refactors and no version skew, but a change under `libs/go/` rebuilds all 15 images.
- 2gRPC inside, HTTP/JSON at the edge: 13 proto domains with a drift check, while auth-gateway to IAM is still HTTP and every cataloged interaction is `retry: none`.
- 3Per-service schema, no cross-service table access: clear ownership, so consistency across services needs outbox and idempotency keys instead of joins.
- 4Only `workflow-service` talks to Zeebe, domains expose `Check*`/`Resolve*` commands: BPMN stays out of domain code, at the price of 117 worker topics to keep in sync with 44 BPMN files.
Fifteen services, very unevenly sized
A diagram of 15 equal boxes would be wrong, so I started with a count. arda-be is one Go workspace (go.work, Go 1.27.2) with 15 modules in apps/. Lines are non-test, non-generated Go from a shallow clone; "first migration" is the timestamp in the earliest goose file name (the clone has one commit, so git cannot give me history).
| Service | Go lines | Test files | First migration |
|---|---|---|---|
| ai-service | 23,509 | 77 | 2026-08-25 |
| workflow-service | 22,428 | 39 | 2026-06-28 |
| loan-service | 12,958 | 33 | 2026-09-06 |
| iam-service | 11,931 | 11 | 2026-06-25 |
| finance-service | 9,602 | 22 | 2026-06-26 |
| statistical-service | 8,343 | 18 | 2026-09-07 |
| deposit-service | 6,657 | 5 | 2026-09-07 |
| notification-service | 5,594 | 18 | 2026-06-29 |
| platform-service | 5,533 | 17 | 2026-06-27 |
| auth-gateway | 4,747 | 13 | no database |
| crm-service | 4,728 | 4 | 2026-06-28 |
| media-service | 3,491 | 7 | 2026-06-27 |
| capital-service | 2,881 | 2 | 2026-09-07 |
| hrm-service | 2,109 | 5 | 2026-07-04 |
| mdm-service | 1,903 | 2 | 2026-09-06 |
Four services (ai, workflow, loan, iam) hold about half of the roughly 126,000 lines. capital and mdm have migrations and gRPC transports but are under 3,000 lines with two test files each; I would call them thin. The dates show the order of arrival: iam, platform, finance, media, crm, workflow, notification, hrm in June and July; ai in late August; mdm, loan, deposit, capital, statistical on 6 to 7 September.
The ownership rule is in docs/refactor-program/01-architecture-and-governance.md: a service owns its rules, schema, migrations, contracts and events, and never queries another service's database. The repo does not record why the original domains were cut this way (no ADR for them). The rule is held by convention and a few CI checks, not by the language or the database.
Three ways services talk, and where each stops
docs/backend-roadmap.md states the rule: HTTP/JSON at the browser edge, gRPC between services, NATS for asynchronous side effects.
Edge. Per arda-infra/docs/api-edge-routing.md, the shell loads from arda.io.vn (Cloudflare Workers static assets) and the browser calls api.arda.io.vn, a Cloudflare Tunnel to auth-gateway:8080. Traefik has explicit routers for iam, mdm, platform and workflow with forward-auth; other services are reached through auth-gateway's own proxy routes. Identity at this edge is headers (X-User-Id, X-Tenant-Id, X-Auth-Version; docs/auth-user-context-contract.md) set by the gateway and never taken from the browser. Making them trustworthy is covered in Authentication beyond JWT.
gRPC. proto/arda/ has 13 domain directories and 14 .proto files; generated Go is committed in libs/go/arda-proto and scripts/check-proto.mjs fails CI on drift. Calls use mTLS plus a signed workload assertion keyed by ARDA_SERVICE_AUTH_SECRET. The call policy is a checked file:
// contracts/interactions/arda-interactions-v1.json
"policy": {
"retry_default": "none",
"deadline_required": true,
"identity_required": true,
"trace_context_required": true
},
"interactions": [{
"id": "auth-gateway.iam-http",
"protocol": "http", "path_prefix": "/internal/iam",
"timeout_ms": 10000, "retry": "none",
"identity": "signed-service-token"
}]
It lists 9 interactions and all say retry: none. Auth-gateway to IAM is still HTTP; the roadmap calls moving it to gRPC "deferred, not in progress". The file covers the IAM adapter and some gRPC calls, not every workflow-to-domain call, so it is a floor, not an inventory.
NATS. contracts/events/arda-events-v1.json registers 16 subjects: 13 wired (11 with a notification-service durable consumer, 2 with a publisher and a reference-only consumer) and 3 draft. Producers: workflow 8, loan 3, platform 2, finance, iam, notification 1 each. Delivery is a JetStream durable consumer acking after the local commit. The producing side is in Reliable domain events; note only that outbox tables exist in loan, workflow, statistical, notification, media and finance, not in every service.
What the newer services changed
Loan, deposit, capital and statistical (and mdm) landed within two days of each other in migration history. They changed the system in two ways a "domain services behind a gateway" picture does not show.
Domains became targets for workflow workers. Each exposes a gRPC *CommandService of Check* and Resolve* calls (plus Settle* for money-moving flows) that Zeebe job workers in workflow-service invoke:
// proto/arda/loan/v1/loan.proto (trimmed)
service LoanCommandService {
rpc CheckDisbursement(CheckDisbursementRequest) returns (CheckDisbursementResponse);
rpc SettleDisbursement(SettleDisbursementRequest) returns (SettleDisbursementResponse);
rpc ResolveDisbursement(ResolveDisbursementRequest) returns (ResolveDisbursementResponse);
rpc CheckCollection(CheckCollectionRequest) returns (CheckCollectionResponse);
// ... batch, provisions, formation, adjustment
}
deposit follows the pattern; capital and statistical expose one Check/Resolve pair each so far. internal/worker/registered_topics.go lists 117 job topics, and a bootstrap test cross-checks them against the serviceTask topics in the 44 embedded .bpmn files. Its comment names the failure it prevents: a case hanging forever at its terminal APPROVE/REJECT step because a process references a topic with no handler. The rule that only workflow-service talks to Zeebe is in BPMN workflow-service boundary; I checked that zeebe_addr appears only in its config, and loan's three "Zeebe" mentions are comments.
Money crossing services made idempotency a schema concern. loan_outbox has dedupe_key VARCHAR(255) NOT NULL UNIQUE, publish_attempts, last_error, and a partial index WHERE published_at IS NULL (apps/loan-service/migrations/20261009090100_loan_disbursement_outbox.sql). finance-service offers a PostingService (ValidatePosting, PostTransaction, ReservePosting, ReleasePosting, ReverseTransaction), docs/loan-contract-exposure.md has the loan side creating a disbursement row and its reservation in one transaction, and deposit has dpm_transactions_idempotency and dpm_interest_posting_states migrations. The shared requirement: a duplicate delivery or retried worker job must not post twice.
ai-service is the odd one. It is the largest module (about 23,500 lines, 25 migrations, its own ai database with pgvector). docs/ai/human-in-the-loop.md has a rule I like beyond AI: a confirm-kind tool cannot decide whether it runs. The sandbox engine marks methods RequiresApproval (where entry.Kind == "confirm") and raises an approval before the dispatcher executes; an independent approver decides, self-approval is rejected, only the run owner can resume, and an unavailable approval store means approval_unavailable, not a silent run. The doc is candid about scope: the only confirm tool, crm.exportCustomer, still creates no export artifact, and real side effects stay off until an owning-domain executor is approved.
From push to pod, and what the cluster is
The path is written in docs/ghcr-k3s-deployment.md and AGENTS.md: a push to main runs .github/workflows/images.yml, which builds one image per changed service (the matrix is computed from changed paths, except the shared-library case above) and pushes ghcr.io/arda-labs/arda-be/<service> tagged main, the git tag, and the commit SHA. In arda-infra, Argo CD Image Updater (writeBackConfig.method: argocd) rewrites image digests, and the arda-backend Application syncs k8s/ into the arda-app namespace with prune and selfHeal on. AGENTS.md adds the caveat I would put in a runbook: a green CI build does not mean pods updated; compare the pinned sha256: digest on the Deployment first.
What the infra repo actually declares, not what I remember of it:
- Cluster. The only dated evidence for node count is
docs/refactor-program/runtime-evidence-20260825.md:k3s-node1,k3s-node2,k3s-node3,Readyonv1.35.5+k3s1on 25 August. That is a snapshot, not a guarantee about today. - Postgres.
database/pg-main.yamlis a CloudNativePGClusterwithinstances: 3, imagepostgresql:18.0, pod anti-affinity by hostname, and 10Gi oflocal-pathstorage per instance. Each service has its own DSN variable (13*_DATABASE_DSNkeys in the secret recipe, plus theaidatabase) and runs its goose migrations at startup. - Namespaces.
docs/deployment-namespace-layout.md:database,platform(NATS, Garage, Valkey, Zeebe),auth(Hydra, Kratos),arda-app,arda-web; no namespace per microservice yet.
There is no backup or archive configuration in pg-main.yaml, and local-path says nothing about disk type; the August evidence found no Backup/ScheduledBackup resources in the database namespace.
What is rough or not done
This list is taken from the repo's own docs, which are unusually willing to say what is unfinished. Dates matter here; I note them.
- Docs disagree with the tree.
README.mdandbackend-current-state.mdsay "11 proto domains".proto/arda/has 13 directories; capital and statistical are not in that count. - Tenant isolation is application-level.
docs/refactor-program/phase-0/rls-pilot.mdsays repository predicates remain the primary control, and that Postgres row-level security is only a pilot on a scratch table, "production adoption is not approved". The 2026-08-25 runtime evidence also counted 54 notification rows, 100 media rows and others with an unassigned tenant value awaiting an owner. I have no later count. - Runtime lagged source, at least on 25 August. That snapshot recorded HRM pods in
CreateContainerConfigError(missing secret key), one of four auth-gateway pods crash-looping, noarda-grpc-tls-*secrets (so the mTLS work was in source but not deployed), and no Arda notification stream or consumer in JetStream. It is a dated snapshot; the repo does not record a later one, so I will not say whether these are fixed. - Events are mostly reference. 3 of 16 registered subjects are draft, and the registry warns that payload schemas must exist before a consumer is enabled. Replay and projection-rebuild evidence are listed as open.
- Retry policy is "none". All 9 cataloged interactions say
retry: none; the progress doc says retry is not uniformly tied to idempotency. - Workflow gaps (as of 2026-09-07).
backend-current-state.mdlists the runtime task claim/complete/reassign facade and incident/retry/suspend/resume APIs as not done; some worker code has landed since, so read the code before trusting that list. - Performance is unmeasured in the repo.
perf/contains a k6 script and a README that saysTARGET_RPS"is the requested rate, not proof that the system achieved that rate". No results are committed. The August evidence has edge time-to-first-byte samples for unauthenticated 401 responses only, explicitly not application or database measurements. That is why this article quotes no latency number.
What I would check next
Whether 15 deployables are justified for the small ones. The namespace doc argues against splitting a namespace until RBAC, network or secret needs differ; the same test applied to mdm, hrm and capital is not recorded anywhere. The repo does not state team size, so I cannot say whether that is a cost. I would also move the library coupling from v0.0.0 plus replace to something that fails loudly when a change to arda-http touches a service's contract, and I would add the missing backup resource before adding another service.
! careful
The runtime findings are from a 2026-08-25 snapshot in `docs/refactor-program/runtime-evidence-20260825.md`. Treat them as history, not current state.
Verified repository evidence
backend · direct
Go workspace of 15 services (arda-be) with internal gRPC between services and shared libraries under libs/go.
↳ 15 Go modules in go.work; protobuf contracts under proto/arda; shared libs under libs/go. Only some libraries are used by every service.
arda-labs/arda-be ↗auth · direct
Layered auth boundary: auth-gateway BFF with Redis-backed sessions behind an arda_sid cookie, Ory Kratos/Hydra for login, and mTLS plus short-lived signed workload assertions between services.
↳ Gateway strips client-supplied identity headers and sets them from the validated session; it signs a short-lived service token for internal calls.
arda-labs/arda-be ↗events · direct
Designed reliable event-driven domain communication using NATS JetStream and the Transactional Outbox pattern to eliminate dual-write hazards.
↳ Envelope versioning (`ardaevents.Envelope[T]`), transactional outbox table writes within SQL transactions, background relay workers, and idempotent consumer deduplication.
arda-labs/arda-be ↗
Key Engineering Takeaways
- Count before you draw: 15 services range from about 1,900 to 23,500 lines of Go, and the small ones are not equal to the large ones.
- Shared libraries made the monorepo consistent, and also made independent release hold only for service code.
- The repo's own docs are the best source of what is not done; reading them changed what this article claims.
- Dated runtime snapshots and committed benchmarks are the only things I would quote for operations and speed; this repo has the first and not the second.
