The Magic Agents

The constraint

Every “AI chatbot for WhatsApp” product I looked at made the same trade: your conversations, your customer records and your prompt all live in someone else’s account. For the businesses I was talking to — clinics, law firms, a real-estate developer — that was the reason they weren’t buying.

So the requirement was isolation that a non-technical buyer could believe in, at a price that works when your customers pay in Colombian pesos.

What I chose

One Postgres, row-level security, no exceptions. Every table carries a tenant_id, and every connection sets it before it can read a byte. A shared database is dramatically cheaper than a database per tenant, but only if isolation is enforced by the engine rather than by the discipline of whoever wrote the last query. RLS moves that guarantee below the ORM.

Go for the edges, Python for the brain. The message gateway, webhook fan-out, rate limiting and billing live in Go, where I want predictable memory and goroutine-per-conversation concurrency. The agent itself is a Pydantic AI service reached over gRPC, because the Python AI ecosystem is where the good tooling actually is. The protobuf boundary is the honest one: it forced me to define what an “agent turn” is instead of letting the two halves leak into each other.

pgvector instead of a vector database. Retrieval volume per tenant is small — a few thousand chunks of knowledge base. Adding a second stateful system to operate would have cost more than the query performance it bought.

Langfuse from day one. Every turn is traced: prompt, tool calls, tokens, latency, cost, tenant. When a customer says “the bot answered badly yesterday afternoon,” I can find that exact turn.

The part that was harder than expected

WhatsApp’s delivery semantics. Messages arrive out of order, duplicate on retry, and a user will happily send four messages in six seconds that together form one question. The fix was a Redis-backed debounce window per conversation plus idempotency keys on inbound webhooks — unglamorous, and the single biggest jump in perceived quality the product ever had.

What I would do differently

I built the channel abstraction after WhatsApp rather than before it, so the interface has WhatsApp’s assumptions fossilised into it. Adding Telegram was easy; adding the embeddable web widget meant admitting that “phone number” is not a universal identity and reworking the contact model under live traffic.