ealeng
Menu

Episode 01 of 7

How a multi-tenant MCP server works

One MCP server, shared by many AI clients and many customer organizations. Follow one request through OAuth, per-tenant rate limits, tool routing and scopes, Postgres row-level security, audit logs, and the error-rate monitor.

MCP Server · Multi-Tenant Architecture8 min read
Video transcript5:04

0:00Overview

0:01

This diagram shows a multi-tenant MCP server.

0:04

One MCP server, shared by many clients and many customer organizations.

0:10

Every call is authenticated, rate limited, routed, isolated by tenant, and observed.

0:16Meet Becky

0:18

Meet Becky. She’s a sales rep at Acme, and she’s been fighting the CRM all morning.Becky: Ugh. This CRM, again… 😩

0:23

So she asks her AI assistant: find Dana Kim at Northwind, and add her if she’s not in the CRM.Becky: Find Dana Kim at Northwind, and add her if she’s not in the CRM.

0:30

Let’s follow her request through the server.

0:3301 · Clients & endpoint

0:35

Becky’s assistant is an MCP client. Requests like hers come from many clients: Claude, ChatGPT, IDEs, and in-house agents.Becky: Find Dana Kim at Northwind, and add her if she’s not in the CRM.

0:45

They hit the CRM MCP server over stateless Streamable HTTP, so any instance can serve any request, and the server can scale out horizontally.

0:5602 · Where the token comes from

0:57

Every call needs an OAuth access token. So where does it come from? Not from this server. The MCP server never creates tokens. It only checks them.

1:07

The first time a client connects without a token, the server answers with a 401, and tells it which authorization server to use.

1:16

That’s the sign-in prompt Becky sees. She signs in to Acme CRM, and approves access.Sign-in prompt: (on screen)

1:22

The authorization server then issues an access token, which carries her organization as an org_id claim, and the scopes she granted, like crm:read.

1:32

From then on, the client sends that token with every request.

1:3603 · Authentication

1:38

On each call, the OAuth step validates the token: its signature, its issuer, that it was issued for this server, and that it hasn’t expired.

1:46

Then it reads the tenant from the org_id claim, never from the tool arguments, so a client can’t ask for another customer’s data.

1:55

A missing, expired, or invalid token is rejected with a 401, which goes straight back through the server to the client.

2:0304 · Per-tenant rate limits

2:05

Next come per-tenant rate limits. They sit after OAuth, because you can only count calls per tenant once you know who the tenant is.

2:13

120 calls a minute, with a burst of 20.

2:16

One noisy customer can’t starve everyone else. Throttled calls get a 429.

2:2305 · Tool routing

2:24

The tool router dispatches each call by tool name to one of three exposed tools.

2:29

Unknown tools fall through to the default branch, and get an error back through the server.

2:3506 · Exposed tools

2:36

Each tool declares an input schema, the OAuth scope it needs, and safety hints.

2:41

search_contacts needs crm:read, and is marked read-only.

2:4606 · Exposed tools

2:47

create_contact needs crm:write. A token without that scope gets a 403. Its description also tells the model to search first, to avoid duplicates. Which is exactly what Becky asked for.

3:0206 · Exposed tools

3:03

delete_contact is also marked destructive, so well-behaved clients ask the user before calling it.

3:10

But hints are only hints. The real protection is the crm:write scope, which the server enforces.

3:1707 · Tenant-isolated data

3:18

All three tools read and write the CRM database: Postgres with row-level security on tenant_id.

3:26

Even a buggy tool can’t read another tenant’s rows.

3:3008 · Results

3:31

Results flow back to the server, which returns them to the client as JSON. Every error takes the same path.

3:38Back to Becky

3:40

Back to Becky. The results reach her assistant, and it replies: Dana wasn’t in the CRM, so I added her.Assistant: Dana Kim wasn’t in the CRM, so I added her. ✓

3:47

One search, one create, her tenant checked at every step, and Becky never had to think about any of it.Becky: Wait, that’s it? That was easy! 🎉

3:5309 · Audit log

3:55

Every write is audited.

3:57

Every create and delete attempt, including scope denials, records the tenant, client, tool, arguments, and result.

4:0410 · Tracing

4:06

Finally, operations. Tracing records every step of every call, across the whole flow, with personal data redacted.

4:1411 · Error-rate monitor

4:15

The error-rate monitor watches every response the server sends back, for all tenants.

4:21

It divides failed responses by all responses, over a rolling five-minute window.

4:26

Failures include auth rejections, rate-limit hits, unknown tools, and tools that fail, for example when the database is down.

4:34

If more than 2 percent of responses fail, it pages on-call.

4:38

The type of error points to the cause. A spike of 401s often means a broken token rollout. A wall of 429s, a runaway client. And tool errors, a failing backend.

4:52Summary

4:53

Authenticate. Limit. Route. Isolate. Audit. Observe.

4:57

That’s how one MCP server safely serves many clients and many tenants.Becky: Thanks! 😄

Key takeaways

  • ✓The MCP server never creates tokens; it only checks them, on every call: signature, issuer, audience, expiry. Anything wrong is a 401 that points to the authorization server.
  • ✓The tenant comes from the token's org_id claim, never from tool arguments. A smuggled org_id is stripped, and the row still lands in the caller's tenant.
  • ✓Rate limits come after authentication, because you can only count per tenant once you know who the tenant is: 120 calls a minute, bursts of 20, then 429 with Retry-After.
  • ✓Tool hints like readOnly and destructive are only hints. The server enforces scopes (403), and Postgres row-level security enforces the tenant even if a tool's query is buggy.
  • ✓Audit every write attempt, denials included, and count every response, including tool errors that come back as HTTP 200. Above a 2% error rate, page on-call.

A multi-tenant MCP server lets many customers' AI clients call the same tools, so every request goes through the same checks, in the same order: who is calling, how fast, which tool, which tenant's data, and what gets recorded. This chapter follows one request through those checks with what each one returns, how the acme-crm-mcp demo implements them (TypeScript MCP SDK, Express, Postgres), and how its tests prove them.

Try it · Follow identity through the serverMCP series · trust boundaries

Identity issuer

OAuth authorization server

Issues a signed token with org_id=acme and crm:read.

Bearer token

Request identity

A user at Acme · acme tenant

The client sends a tool call. Any org_id in its arguments is ignored for identity.

MCP server · trusted request path

tenant = token.org_id

01 · Validate

Signature · issuer · audience · expiry

02 · Limit

120/min · burst 20 · per tenant

03 · Authorize

search_contacts needs crm:read

04 · Query

Set tenant context in transaction

tenant_id = acme

Database guard

PostgreSQL row-level security

Returns Acme rows only, even if a query misses its tenant filter.

Tool result

Scoped response to client

Data returned through the same authenticated request.

Two independent checks protect tenant data: the server derives identity from the validated token, and Postgres enforces that tenant at the row boundary.

The checks at a glance

Stage What it checks On failure In the demo
Authenticate Token signature (authorization server's public keys), issuer, audience = this server, expiry; tenant from the org_id claim 401, with WWW-Authenticate pointing to the protected-resource metadata src/pipeline.ts
Rate limit The tenant's token bucket: 120 calls/minute, bursts of 20 429 with Retry-After src/pipeline.ts (in memory, per instance)
Scope The tool's required scope: crm:read or crm:write 403 insufficient_scope; a denied write is audited src/pipeline.ts
Route A registered tool name, and arguments that match its schema Tool error: HTTP 200 with isError: true MCP SDK
Data Postgres row-level security on tenant_id Another tenant's rows don't exist for this request db/01_schema.sql, src/db.ts
Observe Every write attempt audited; every response counted Page above 2% errors over 5 minutes audit_log, src/observability.ts, /metrics

The order matters. Rate limiting needs a tenant, so it comes after authentication, and a call with a bad token never spends any tenant's budget. Scopes come before the tool runs, so a denied write never touches the database.

Try it: send a request

Pick a token and a call. The simulator follows the demo's code and seed data: tenant acme has Priya Shah, Marcus Lee and Sofia Alvarez; tenant globex has Dana Kim (c_201) and Tom Becker.

Try it · Send a request through the serverBehaviour of the acme-crm-mcp demo
Token

org_id=acme · crm:read crm:write

Call

tools/call search_contacts {"query":"Dana"}

  1. 1 · Authenticate

    ✓ pass

    signature · issuer · audience · expiry ✓ → tenant = acme (from org_id)

  2. 2 · Rate limit

    ✓ pass

    acme's bucket has a token

  3. 3 · Scope

    ✓ pass

    crm:read ✓

  4. 4 · Tool + Postgres RLS

    ✓ pass

    SET app.tenant_id = 'acme' · RLS shows 3 of 5 rows

Response · HTTP 200

{"matches": 0, "contacts": []}

Database · Dana Kim exists, as c_201, but in globex. RLS hides that row from acme entirely.

audit_log · no row (only write attempts are audited)

/metrics · mcp_responses_total{tenant="acme", outcome="ok"}

Worth trying:

  • acme-agent → search "Dana": 0 matches. Dana exists, but in globex, and row-level security hides that row from acme. That's why creating her in acme is the right outcome for the request in the video.
  • acme-agent → create with org_id: globex: the row lands in acme. The token decides.
  • acme-agent → delete c_201: "No contact with id c_201", and the row is untouched in globex. Another tenant's id looks exactly like a missing one.
  • acme-readonly → create: 403, and an audit row with outcome denied.
  • drop_all_tables: HTTP 200, but a failure. See Counting errors.

Tokens: discovery and validation

The MCP server never creates tokens; it only checks them. A client's first call has no token, and the 401 tells it where to go:

401  WWW-Authenticate: Bearer resource_metadata="http://localhost:8787/.well-known/oauth-protected-resource"

GET /.well-known/oauth-protected-resource
{ "resource": "http://localhost:8787/mcp",
  "authorization_servers": ["http://localhost:8788"],
  "scopes_supported": ["crm:read", "crm:write"] }

The client signs the user in at that authorization server and gets back an access token with an org_id claim and the granted scopes, then sends it on every call. The server checks four things, every time:

Check Catches
Signature, against the authorization server's public keys (JWKS) A token anyone else minted, even with the right claims
Issuer A token from a different authorization server
Audience = this server A token issued for some other API, replayed here
Expiry An old token

Decoding a token isn't verifying it: the claims are readable by anyone, which is why the signature check comes first. The demo's authorization server is a mock that issues tokens with client credentials (acme-agent, acme-readonly, globex-agent). A production one (Auth0, Okta, Keycloak) adds the authorization-code flow with PKCE, so clients like Claude Desktop can sign real users in.

The tenant comes from the token

The tenant is the token's org_id claim, never a tool argument. The demo enforces that three ways:

  • No tool has a tenant argument, and each tool's input schema strips arguments it doesn't declare, so a smuggled org_id never reaches the code.
  • The database gets the tenant from the request, set per transaction: SELECT set_config('app.tenant_id', $1, true). The true makes it local to the transaction, so it can't leak to the next request on the same pooled connection.
  • Inserts take it from that setting (current_setting('app.tenant_id')), not from the arguments.

Rate limits per tenant

A token bucket per tenant: it starts with 20 tokens (the burst) and refills at 120 a minute, 2 a second. Each call takes one; an empty bucket is a 429 with Retry-After in seconds. One tenant flooding the server empties only its own bucket.

The demo's buckets live in memory, per instance. Behind a load balancer with several instances, each would count separately, so shared state (Redis) is the production step.

Scopes enforce; hints guide

Tool Scope Hints Description tells the model
search_contacts crm:read readOnlyHint Call this before create_contact to avoid duplicates
create_contact crm:write not read-only, not destructive Only call this after search_contacts finds no match
delete_contact crm:write destructiveHint Confirm with the user first

Descriptions and hints shape what a well-behaved client does: search before creating, ask before deleting. A client can ignore them. What it can't get past is the scope check on the server, and the database policy under it.

Row-level security that survives buggy queries

ALTER TABLE contacts ENABLE ROW LEVEL SECURITY;
ALTER TABLE contacts FORCE ROW LEVEL SECURITY;
CREATE POLICY tenant_isolation ON contacts
  USING (tenant_id = current_setting('app.tenant_id', true))
  WITH CHECK (tenant_id = current_setting('app.tenant_id', true));

Three details make it hold:

  • The app connects as app_user, which doesn't own the table, and RLS is FORCEd. Table owners bypass RLS by default.
  • USING filters reads, WITH CHECK filters writes, so a tool can neither see nor create another tenant's rows.
  • An unset tenant matches nothing: current_setting(..., true) returns NULL, so a request that forgot to set it sees no rows at all, instead of every row.

The audit log has the same policy.

Audit and tracing

Every create and delete attempt goes to audit_log: tenant, client, tool, arguments, outcome (ok, denied, error). Denials matter as much as successes: a read-only integration trying to write is exactly what you want to find. A call rejected at authentication isn't audited, because there's no validated tenant to file it under; it shows up in the metrics instead.

The demo writes one structured log line per request (tenant, method, tool, status, outcome, duration), and deliberately leaves the arguments out, since they can contain personal data. That line is where OpenTelemetry tracing would go in production.

Counting errors

The error-rate monitor is failed responses ÷ all responses, for all tenants, over a rolling 5 minutes; above 2%, page on-call. The demo exposes the counts at /metrics (example counts):

mcp_responses_total{tenant="acme",outcome="ok"} 41
mcp_responses_total{tenant="acme",outcome="403"} 1
mcp_responses_total{tenant="acme",outcome="tool_error"} 2
mcp_responses_total{tenant="unknown",outcome="401"} 3

The catch: tool errors are HTTP 200. An unknown tool comes back as a result with isError: true and the text "MCP error -32602: Tool drop_all_tables not found"; so do bad arguments and a tool that fails because the database is down. A monitor that only counts HTTP errors misses them, so the demo looks inside the response body and counts them as tool_error.

Spike of Likely cause
401s A broken token rollout
429s A runaway client
Tool errors A failing backend

How the demo proves it

The contract tests (tests/contract.test.ts) run on every commit with no model in the loop, against a real Postgres, and check isolation in the database, not just in the response:

  • 401 for a missing, expired, wrong-audience, or forged token; the metadata names the authorization server
  • 403 for a read-only token on create_contact, with the denial in audit_log
  • 429 for a flooding tenant (the tests use a burst of 5: 5 × 200, then 3 × 429, with Retry-After), while another tenant still gets 200
  • an unknown tool answered with an error, not a crash; the tool list carries its safety hints
  • tool errors counted in /metrics
  • acme can't see globex's Dana, can't delete her by exact id (the row is still in globex), and a smuggled org_id is ignored
  • every successful write audited with tenant, client, tool and arguments

Chapter 2 builds the full test strategy on top of these.

Practice: the request pipeline

Port the server's decisions to Python: token validation that takes the tenant only from the claims, a per-tenant token bucket with Retry-After, the pipeline in order (denied writes audited, bad tokens kept away from the buckets), and an error rate that counts tool errors returned as HTTP 200. The tests are the demo's contract tests.

Try it · Practice in PythonRuns in your browser · ⌘/Ctrl + Enter to run
tenant_practice.py72 lines

Quiz

Try it · Check your understanding0 of 6 answered
  1. 1.A tool call includes org_id: "globex" in its arguments, but its bearer token's org_id claim is acme. Where does the new contact land?

  2. 2.An acme client searches for "Dana" and gets 0 matches, though a Dana Kim at Northwind is in the database. Why?

  3. 3.Why does the per-tenant rate limit sit after authentication, not before?

  4. 4.delete_contact is annotated destructiveHint: true. A client ignores the hint and calls it with a read-only token. What stops it?

  5. 5.The error-rate monitor only counts HTTP responses with a status of 400 or more. What does it miss?

  6. 6.The error rate jumps above 2%, and most failures are 401s. What's the likely cause?

Check your understanding

Try each question before opening its answer.

Where does the MCP server get the token, and what does it check on every call?

From the client, which got it from the authorization server; the MCP server never creates tokens. A first call without one gets a 401 that points to the protected-resource metadata. On every call: signature (the authorization server's public keys), issuer, audience (this server), expiry. Any failure is a 401.

Where should the server get the tenant ID?

Only from the validated token's org_id claim. Tool arguments are untrusted: no tool takes a tenant argument, the schema strips unknown ones, and the database row takes its tenant from the transaction setting.

Why do rate limits come after authentication?

The bucket is per tenant, and the tenant is only known once the token is validated. It also keeps unauthenticated calls from spending any tenant's budget.

Do readOnly or destructive annotations enforce access?

No. They help clients and models behave (ask before deleting). Enforcement is the server's scope check (403) and Postgres row-level security.

What makes the row-level security hold even against a buggy query?

The app connects as a non-owner role with RLS forced, the tenant is set per transaction from the token (and can't leak to the next request), and the policy checks every read and write. Unset tenant means no rows.

Why audit denied writes as well as successful ones?

Denied attempts reveal risky or misconfigured clients, like a read-only integration trying to write. The audit trail keeps the attempt, who made it, and the outcome.

What should the error-rate monitor count?

Every response to every tenant: 401/403, 429, unknown tools and failing tools. Tool errors are HTTP 200 with isError in the result, so the monitor has to look inside the body. Page above 2% over 5 minutes, and use the error type to find the cause.

Next up: how do you test a server like this? Chapter 2 builds five layers, from contract tests with no model in the loop to sampled production review.

Learning something useful?

New labs, articles, and videos on evaluating and testing AI systems come out regularly. Follow along so you don't miss the next one.