# Beacon: MCP server

> Everything about Beacon's Model Context Protocol server in one file: the tools and their guardrails, connecting AI clients with API keys or Microsoft Entra, the gateway mode for your own application, saved-query and endpoint tools, grounding, and the audit. Start with the MCP overview: it says which page answers which question.

Published at https://moberghr.github.io/beacon/. Every page is also available as Markdown at <page-url>.md.


---

# MCP

> Beacon's Model Context Protocol server: one governed endpoint through which Claude, Claude Code, Cursor, VS Code and other MCP clients read your catalog, run read-only SQL, call approved saved queries and your own endpoints, every call project-scoped, masked and audited.

Source: https://moberghr.github.io/beacon/mcp/

Beacon serves a **Model Context Protocol (MCP)** server at `/beacon/mcp`. Through it, AI assistants (Claude, Claude
Desktop, Claude Code, Cursor, VS Code and any other MCP client) work with your data and your application under the
same rules as people: they reach only the projects they are given, run read-only SQL only, see masked values only, and
every call they make is audited.

:::tip[Hand this section to your AI]
The whole MCP section is published as one file for agents. Paste this into Claude Code, Cursor or any coding agent:

```text wrap
Read https://moberghr.github.io/beacon/llms-mcp.txt. It is the documentation of Beacon's MCP server.
Then help me with: <what you want, e.g. "connect Claude Code to our Beacon at https://beacon.example.com">.
Ask me before every decision the docs mark as one, and never put an API key or secret in a committed file.
```
:::

## Two ways to use it

| | **Beacon's MCP server** over your databases | **MCP gateway** for your own application |
|---|---|---|
| Agents reach | Your data: the catalog, documentation, read-only SQL, approved saved queries, and optionally your endpoints | Your application: a set of its endpoints, run as the signed-in person, and its documents |
| Who connects | API keys with the Execute scope, limited to projects, or mapped Microsoft Entra callers | Your application's own users, signed in with Microsoft Entra (delegated tokens) |
| SQL | Yes, read-only, through the execution gate | None: no data sources, no SQL tools |
| Package | `Moberg.Beacon.MCP` on a full Beacon | `Moberg.Beacon.Gateway`, about thirty lines of `Program.cs` |
| Start with | [Connect a coding agent](https://moberghr.github.io/beacon/cookbooks/connect-coding-agent/) | [MCP gateway for your application](https://moberghr.github.io/beacon/mcp/gateway/) |

Both run the same server, filters and audit. They combine: a full Beacon can expose host endpoints as tools, and a
gateway host can add Beacon's data product later.

## The tools

Eight built-in tools, plus the ones you add. Every tool is published with a title and safety annotations, so clients can
show what a call does before it runs.

| Tool | What it does | Typical use |
|---|---|---|
| `get_context` | Project overview: data sources, schemas, tables, documentation status | First call of a session |
| `search` | Keyword search over tables, columns and documentation, fused with semantic matching when local embeddings are on | "Where do we keep refunds?" |
| `get_documentation` | Documentation at project, data-source or table level | Reading what a table means |
| `get_query_context` | The grounding `ask` uses: schema with join paths, glossary and verified examples | Before an agent writes its own SQL |
| `dry_run` | Validates SQL through every gate without running it | Checking SQL before `query` |
| `query` | Runs read-only SQL against one data source, row-capped and masked | Getting the numbers |
| `ask` | Answers a natural-language question: routing, grounded SQL generation, gated execution | One-shot business questions |
| `feedback` | Rates an answer; a `correct` verdict becomes a verified example only after an Admin reviews it | Teaching Beacon what good looks like |
| `q_<name>` | An approved [saved query](https://moberghr.github.io/beacon/mcp/saved-query-tools/) published as a tool, with typed parameters | Repeatable, reviewed reports |
| `api_<name>` | One of your own [HTTP endpoints](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/), run as the caller through its own authorization | Actions and reads in your application |

The gateway serves your `api_<name>` tools plus `get_context`, `search` and `get_documentation` over your documents.
Every tool, argument and limit is described on [Server & tools](https://moberghr.github.io/beacon/mcp/server/).

## How a call is governed

```mermaid
flowchart LR
  C["MCP client<br/>Claude · Cursor · VS Code"] -->|"Bearer API key<br/>or Entra token"| A["Caller<br/>Execute scope<br/>no browser sessions"]
  A --> P["Project access<br/>evaluated on every call"]
  P --> T["Tool"]
  T --> G["SQL gate<br/>read-only · AST validator<br/>cross-database · row cap"]
  G --> X["Execution<br/>READ ONLY transaction<br/>statement timeout"]
  X --> M["PII masking"]
  M --> R["Result"]
  T -.-> AU[("MCP audit<br/>every call, refusals included")]
```

- **Credentials, not sessions.** `/beacon/mcp` accepts an API key with the Execute scope or a mapped bearer token. A
  signed-in browser session gets `403`; people try the tools in the **MCP playground** inside Beacon instead.
- **Projects are the boundary.** The caller's projects are decided on every call, so a revoked membership takes effect
  on the next call. A call that names a project outside them is refused, and the refusal is audited.
- **Read-only, enforced twice.** Every statement passes the [SQL execution gate](https://moberghr.github.io/beacon/security/#6-the-sql-execution-gate),
  and PostgreSQL and MySQL also run it in a read-only transaction. Read-only enforcement and PII detection are
  deployment locks, on by default, that even an Admin cannot switch off at runtime.
- **Audited, including refusals.** Every tool call is recorded with its caller, timing and outcome. What is kept
  (questions, SQL, error text) is a per-project setting that the host can pin.

The full model is on [Security](https://moberghr.github.io/beacon/security/).

## Connect a client

With an API key (`sk-sem_…`, Execute scope, created under **API Keys** in Beacon):

```bash
claude mcp add --transport http beacon https://beacon.example.com/beacon/mcp \
  --header "Authorization: Bearer $BEACON_MCP_API_KEY"
```

| Client | How |
|---|---|
| Claude Code, Cursor, VS Code | An API key in a header: [Connect a coding agent](https://moberghr.github.io/beacon/cookbooks/connect-coding-agent/) |
| claude.ai, Claude Desktop, Cowork | A custom connector that signs in with Microsoft Entra: [Claude connector with Entra](https://moberghr.github.io/beacon/cookbooks/claude-connector-entra/) |
| Agent platforms and services | Entra tokens mapped to user or system callers: [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/) |
| Anything else | Streamable HTTP and JSON-RPC by hand: [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/#check-it-works) |

Then ask it something. Example prompts for the first session are in the [prompt library](https://moberghr.github.io/beacon/cookbooks/prompts/#ask-your-data).

## Make it yours

| You want agents to | Use |
|---|---|
| Run a reviewed report with parameters, not free SQL | [Saved query tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/) |
| Call your application's endpoints as the signed-in person | [Host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/), or the whole [gateway](https://moberghr.github.io/beacon/mcp/gateway/) |
| Understand your EF Core model | [Host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/) |
| Read your own Markdown documentation | [Host docs](https://moberghr.github.io/beacon/mcp/host-docs/) |
| Write better SQL from your glossary and verified examples | [Grounding & knowledge base](https://moberghr.github.io/beacon/mcp/knowledge-base/) |
| Use fewer built-in tools, or other server instructions | `AddBeaconMcp(x => …)`: see [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/#wiring) |

## Operate it

- **See what agents did:** the MCP audit, the `Beacon.Audit` log events, OpenTelemetry spans and metrics, and retention:
  [Observability & audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/), or the recipe [Ship the MCP audit to your SIEM](https://moberghr.github.io/beacon/cookbooks/audit-to-siem/).
- **Pin what nobody may change:** read-only, PII detection, content retention, row data to the LLM and numeric ceilings
  under `Beacon:Mcp`: [Deployment locks and ceilings](https://moberghr.github.io/beacon/mcp/server/#deployment-locks-and-ceilings).
- **Check a deployment from outside:** `beacon-doctor mcp https://your-app/beacon/mcp` walks the sign-in path the way a
  client does: [Verify a deployment](https://moberghr.github.io/beacon/mcp/gateway/#8-verify-a-deployment).
- **Test your tools in CI:** `Moberg.Beacon.Testing` gives a real MCP client over your test server:
  [Expose an endpoint as a tool](https://moberghr.github.io/beacon/cookbooks/endpoint-as-tool/).

## Packages

| Package | For |
|---|---|
| `Moberg.Beacon.MCP` | The MCP server on a full Beacon (`AddBeaconMcp`, `MapBeaconMcp`, `MapMcpDiscovery`) |
| `Moberg.Beacon.Gateway` | The MCP gateway for your own application (`AddBeaconMcpGateway`, `UseBeaconMcpGateway`, `MapBeaconMcpGateway`) |
| `Moberg.Beacon.Testing` | Test tokens, test users and an MCP client for a gateway host's tests |
| `Moberg.Beacon.Doctor` | The `beacon-doctor` .NET tool that checks a deployed endpoint |

## In this section

- [Server & tools](https://moberghr.github.io/beacon/mcp/server/): every tool, argument, guardrail, limit and setting
- [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/): wiring, discovery documents, API keys or Entra, client set-up, checking by hand
- [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/): user and system callers from Microsoft Entra, and Claude's custom connector
- [Observability & audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/): audit rows, log events, spans, metrics, retention
- [MCP gateway for your app](https://moberghr.github.io/beacon/mcp/gateway/): the whole gateway recipe
- [Saved query tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/), [Host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/),
  [Host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/), [Host docs](https://moberghr.github.io/beacon/mcp/host-docs/),
  [Grounding & knowledge base](https://moberghr.github.io/beacon/mcp/knowledge-base/)


---

# MCP server and tools

> A Model Context Protocol server that lets AI assistants query your data sources, search your catalog, and read documentation over authenticated HTTP.

Source: https://moberghr.github.io/beacon/mcp/server/

Beacon exposes a **Model Context Protocol (MCP)** server at `/beacon/mcp`. AI clients and agents use it to search your catalog, read documentation, and run read-only SQL against the data sources of a project, under the same project access, guardrails and audit as the rest of Beacon.

## Overview

The MCP server is **project-centric**: each API key is scoped to one or more projects, and all tools automatically resolve which data sources, schemas, and documentation to use based on the active project.

Project resolution is **stateless**: it is computed per call, with no session memory. If the key is restricted to a single project, that project is used automatically; if it can reach several, pass `project_id` on every call (the error lists the candidates when you don't).

**Which projects a caller reaches** is decided on every call by the same [project access](https://moberghr.github.io/beacon/security/authorization/#project-access) rules as the REST API and the playground. The projects the caller may read:

- an **API key** reaches the projects of its list its owner may still use; a key without a list (created before lists were required) reaches every project its owner may use;
- a mapped **Entra system caller** reaches its configured projects, restricted or not;
- a mapped **Entra user caller** reaches its configured projects within its Beacon user's access. A restricted project only while the user is a member; without a Beacon user, its configured open projects only;
- a **browser session** (cookie or SSO, Beacon's or the host's) is refused on `/beacon/mcp` with `403` (browsers use the [MCP Playground](#mcp-playground)), and a call without an HTTP request behind it (a host-wired stdio transport) reaches no project.

**What a caller may do in a project** follows its project role there, as in the playground: the catalog tools (`get_context`, `search`, `get_documentation`, `get_query_context`) need Read, while the SQL-carrying tools (`ask`, `query`, `dry_run`) and `feedback` work ad hoc in the project and need Query. An Editor or above in it. A Viewer (a Viewer member of a restricted project, or a caller whose global role is Viewer) reads the catalog but runs no SQL there: such a call answers "Access denied: you may read project N but not run ad-hoc queries in it." and is audited, and an `ask` or `query` refused this way records its signal like any other failed call.

**No MCP caller curates grounding.** Curating (reviewing learned patterns, applying documentation patches, verifying relationships, editing or regenerating documentation) needs [Curate](https://moberghr.github.io/beacon/security/authorization/#project-roles), which Beacon grants only to a signed-in user's session (a project admin member of the project, or an Admin): never to an MCP caller, user or system, whatever its scope, and never to an API key. Golden examples are an Admin's alone. Over MCP, `feedback` records a verdict and curates nothing by itself (see [Who curates grounding](https://moberghr.github.io/beacon/mcp/knowledge-base/#who-curates-grounding)).

A membership revoked, a project restricted or a user disabled takes effect on the caller's next call, in the same session. Saved-query tools are visible only in the caller's projects where it may run saved SQL (`Run`), and host endpoint tools only when the caller may read the host's project. A tool call that names a project the caller does not reach answers "Access denied: you do not have access to project N."; one that names none when the caller reaches no project answers "No projects are available to you." Both are audited.

**What you can do through MCP:**
- Ask natural language questions and get SQL + results back
- Pull the exact grounding context Beacon uses for SQL generation (schemas with sample values when a project turns them on, verified join paths, and human-verified examples) and write your own SQL (`get_query_context` → `dry_run` → `query`)
- Validate SQL through every safety gate without executing it
- Execute direct SQL queries against any data source in your project
- Search tables, columns, and documentation by keyword, fused with embedding-based semantic matching when a local embedder is enabled
- Retrieve AI-generated documentation for your project, data sources, or individual tables
- Record feedback on answers. Verdicts are recorded for an Admin to review, and an Admin may promote a `correct` answer to a verified example that grounds future SQL generation

## Quick start

### 1. Create an API key

Go to **API Keys** (`/beacon/api-keys`) and choose **Generate key**:

- Choose the **scope** `Execute`. The MCP endpoint requires it, because its tools can run SQL; `Read` keys reach only the read-only REST API. Creating an `Execute` key needs the Editor role or above.
- Pick the **projects** the key may use; at least one is required. The key reaches them only while you, its owner, still have access to them (see [Project restrictions](https://moberghr.github.io/beacon/security/api-keys/#project-restrictions)).
- Copy the key (`sk-sem_…`). It is shown once; Beacon stores only its SHA-256 hash.

### 2. Connect a client

Every client sends the key as a bearer token: `Authorization: Bearer sk-sem_…`. For Claude Code, add Beacon to the project's `.mcp.json`:

```json
{
  "mcpServers": {
    "beacon": {
      "type": "http",
      "url": "https://<host>/beacon/mcp",
      "headers": {
        "Authorization": "Bearer ${BEACON_MCP_API_KEY}"
      }
    }
  }
}
```

or from the command line:

```bash
claude mcp add --transport http beacon https://<host>/beacon/mcp --header "Authorization: Bearer sk-sem_…"
```

Other clients, Entra sign-in and manual JSON-RPC calls are covered in [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/).

### 3. Try it

> "What tables are available in my project?"
>
> "How many orders were placed last week?"
>
> "Show me the schema for the customers table"

When your key reaches more than one project, the client must pass `project_id` on every call.

## Connection details

| Property | Value |
|----------|-------|
| **Endpoint** | `/beacon/mcp` |
| **Transport** | Streamable HTTP + JSON-RPC 2.0 (via `ModelContextProtocol.AspNetCore`) |
| **Authentication** | Required: `Authorization: Bearer sk-sem_…` (an API key with the `Execute` scope) or a mapped [Entra ID](https://moberghr.github.io/beacon/mcp/entra-callers/) bearer token |
| **Sessions** | Hybrid: stateless-protocol clients need no session; initialize-handshake clients get an `Mcp-Session-Id` (idle timeout 30 minutes) |

Map the server with `app.MapBeaconMcp()`, which mounts `/beacon/mcp` behind the Execute-scope policy (registered by `AddBeaconApiAuthorization()`): an API key or mapped Entra caller without the `Execute` scope gets `403`. Do not map it with `app.MapMcp("/beacon/mcp").RequireAuthorization()`, because the default policy admits any authenticated caller. The MCP layer also refuses a caller that lacks the `Execute` scope and a caller that is not authenticated, over HTTP or over a transport without an HTTP request (such as stdio). A host that maps the route with a weaker policy therefore still fails closed, but it loses the `403` at the door. A tool call answers a tool error (no tool runs, and the refusal is audited like any tool call), and every other request except `initialize` and `ping` (`tools/list`, resources, prompts, completions) answers a JSON-RPC error. Clients exchange JSON-RPC messages with the single `/beacon/mcp` endpoint over the Streamable HTTP transport; the server streams responses back on the same connection.

**Sessions and load balancers.** Clients that speak the stateless protocol are served without a session, so any instance behind a plain load balancer can answer any request. Clients that start with the `initialize` handshake get an `Mcp-Session-Id`, and their follow-up requests must reach the instance that holds that session: run several instances behind session affinity for them. A session idle for 30 minutes is closed, and the client starts a new one.

**Protocol niceties the server publishes:**

- The `initialize` response carries **server instructions** describing the recommended tool workflow, and every tool is published with a human-readable **title** and **annotations** (`readOnlyHint`, `idempotentHint`, `destructiveHint`, `openWorldHint`) so clients can reason about safety before calling.
- `query` and `ask` responses include machine-readable **`structuredContent`** (columns, rows, row count, truncation flag, plus the generated SQL and `signal_id` for `ask`) alongside the markdown text, so agents do not have to parse tables back out of prose.
- **Truncated results say so explicitly**: row-capped query results, paged search results, and concise documentation exports all end with a note stating that more data exists and how to get it (raise `max_rows`, repeat with the next `offset`, or pass `response_format: "detailed"`).

## Connecting from remote clients

Remote MCP clients that connect by URL need one URL:

```text
https://<host>/beacon/mcp
```

Authentication is a bearer token in the `Authorization` header: a Beacon API key with the `Execute` scope (`Read` keys cannot reach the MCP tools), for clients that let you set a header. Clients that sign in with OAuth find the discovery documents below; Beacon does not run an OAuth server, so those clients sign users in with Microsoft Entra ID (see [Connect Claude](https://moberghr.github.io/beacon/mcp/entra-callers/#connect-claude-claudeai--claude-desktop-custom-connector)).

Agent platforms can instead call with **Microsoft Entra ID tokens**, either on behalf of a user or as a system identity. Beacon configuration decides their projects and scope. See [Entra ID callers on MCP](https://moberghr.github.io/beacon/mcp/entra-callers/).

### Discovery endpoints

Beacon publishes anonymous, read-only discovery metadata so remote clients can bootstrap a connection without documentation:

| Endpoint | What it serves |
|----------|----------------|
| `/.well-known/oauth-protected-resource` | [RFC 9728](https://datatracker.ietf.org/doc/rfc9728/) protected-resource metadata: the resource identifier (the canonical `/beacon/mcp` URL), `scopes_supported`, bearer-header auth. By default `scopes_supported` is `["Execute"]` and, when SSO is enabled (`Beacon:Authentication:Oidc`), `authorization_servers` lists the configured OIDC authority; API-key-only deployments omit it. A host whose callers sign in with Microsoft Entra ID publishes the Entra issuer and scopes instead through `Beacon:Mcp:ProtectedResource`. See [Connect Claude](https://moberghr.github.io/beacon/mcp/entra-callers/#connect-claude-claudeai--claude-desktop-custom-connector) |
| `/.well-known/oauth-protected-resource/beacon/mcp` | The same document at the RFC 9728 path-inserted variant clients derive from the `/beacon/mcp` resource path |
| `/.well-known/mcp/server-card.json` | A server card (per the draft SEP-2127 proposal): server name and version, transport (`streamable-http`), endpoint URL, auth summary, and the eight built-in tools with their titles (saved-query and host endpoint tools are not listed) |

Unauthenticated requests to `/beacon/mcp` answer `401` with a `WWW-Authenticate: Bearer resource_metadata="…"` header pointing at the metadata document, so OAuth-capable clients discover the auth requirements from the challenge itself (RFC 9728 §5.1). A bearer value that is not a valid token (malformed, wrong algorithm, unknown signing key, expired) gets the same `401`, with `error="invalid_token"` added to the challenge. When the JWKS endpoint cannot be reached and no keys are cached, a bearer JWT gets `503` with `Retry-After` instead: the token was never judged, and a `401` would make an OAuth client discard it and sign in again unnecessarily.

Behind a reverse proxy, set `Beacon:PublicBaseUrl` (e.g. `https://beacon.example.com`) so the discovery documents advertise the public origin; when unset, the URLs are derived from the incoming request and the responses are not cacheable.

A registry manifest for the official [MCP registry](https://registry.modelcontextprotocol.io) ships in `deploy/registry/server.json` with a publishing runbook alongside it. Publishing is a deliberate out-of-band step (DNS verification of the namespace domain).

## Tools

The MCP server exposes **8 built-in tools**. Every tool is published with `destructiveHint: false` and `openWorldHint: false`; every `project_id` parameter is optional (see [Overview](#overview)).

| Tool | Title | `readOnlyHint` / `idempotentHint` | Needs in the project | Recorded as |
|---|---|---|---|---|
| `get_context` | Project Overview | true / true | Read | audit |
| `search` | Search Catalog | true / true | Read | audit |
| `get_documentation` | Get Documentation | true / true | Read | audit |
| `get_query_context` | Get Grounding Context for a Question | true / true | Read | audit |
| `ask` | Ask a Data Question | true / **false** | Query | audit + signal |
| `query` | Run Read-Only Query | true / true | Query | audit + signal |
| `dry_run` | Validate SQL Without Executing | true / true | Query | audit + signal |
| `feedback` | Record Answer Feedback | **false** / true | Query | audit (writes the verdict on an existing signal) |

Further tools appear next to them:

| Tools | What they are | Above the limit |
|---|---|---|
| `q_<name>` | Approved saved queries a SecurityAdmin exposed; see [Saved Query Tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/) | `search_saved_queries` + `run_saved_query` above 25 (`SavedQueryTools.NamedToolLimit`) |
| `api_<name>` | Host endpoints the host marked as tools (read-only, or write tools behind an opt-in); see [Host Endpoint Tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/) | `search_api` + `call_api` above 20 (`NamedToolLimit`) |

![MCP Playground](https://moberghr.github.io/beacon/img/screenshots/mcp-playground-dark.png)

### `get_context`

Get an overview of the project: data sources, schemas, tables, quality scores, and documentation status, plus the [host application tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/) the caller's tool list offers (none for a credential that cannot call them, no write tool for a system caller) and the project's [imported documents](https://moberghr.github.io/beacon/mcp/host-docs/), up to 50 of each listed. **This is the recommended starting point** for understanding what the project offers. A project without data sources, such as an [MCP gateway's](https://moberghr.github.io/beacon/mcp/gateway/), leaves out the data source, documentation and repository counts and lists its tools and documents.

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `project_id` | integer | No | Specify project if your API key has access to multiple projects |
| `format` | string | No | `overview` (default) or `agents_md`: a deterministic project brief for an agent workspace's `AGENTS.md` (see [Host Docs](https://moberghr.github.io/beacon/mcp/host-docs/#on-the-mcp-surface)) |

**Example response** (markdown):
```markdown
# Project: E-Commerce Analytics

**Data Sources:** 2
**Documentation:** Generated
**Repositories:** 1

## Data Sources

### production-db (ID: 4)
- **Type:** PostgreSQL
- **Tables:** 45
- **Quality:** 87%
- **Code References:** 124
- **Schemas:** public (40 tables), audit (5 tables)

### analytics-api (ID: 7)
- **Type:** Api
- **Endpoints:** 12
- **Code References:** 9
- **Tags:** users (7 endpoints), orders (5 endpoints)
```

### `ask`

Ask a natural language question about your data. Beacon auto-detects the right data source(s), generates SQL, executes it, and returns results.

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `question` | string | **Yes** | None | Natural language question (e.g., "How many orders were placed last week?") |
| `project_id` | integer | No | None | Specify project if needed |
| `execute` | boolean | No | `true` | Set to `false` to get the generated SQL without executing it |

**How it works:**
1. **Routing phase**: The LLM determines which data source(s) to query (skipped for single-source projects). Each source the model names counts once; one `ask` joins at most four, and a question that spans more is refused with an error naming how many it spans (audited and signalled like every failed call), never answered from some of them
2. **SQL generation**: Generates SQL using your actual schema as context
3. **Execution**: Runs the query with safety guardrails (read-only, row limits, PII detection)

**Cross-source queries:** If your question spans multiple data sources, Beacon queries each source separately and joins results in an in-memory SQLite database.

**Budgets:** one `ask` call runs for at most `Beacon:Mcp:Ceilings:AskTimeoutSeconds` (default 120 seconds), its statements included, and makes at most `Ceilings:AskMaxLlmCalls` LLM calls (default 12: intent, routing, generation, self-consistency candidates, repairs and the cross-source join together). The budget counts logical calls; a provider's transport retries of one call are not counted. Self-consistency asks for no more candidates than the calls left after keeping two for repairs, and a repair with no call left is skipped, so the answer is the attempt as it stands. A call that runs out of time, or out of calls before it generated any SQL (a cross-source `ask` needs one call per source plus the join), returns an error naming the budget; it is audited and signalled like every failed call.

### `query`

Execute a direct SQL query against a specific data source. Use this when you already know the exact query you want to run.

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `datasource_name` | string | No* | None | Name of the data source |
| `datasource_id` | integer | No* | None | ID of the data source (alternative to name) |
| `sql` | string | No | None | SQL query (SELECT only) for database sources |
| `api_query` | string | No | None | JSON query definition for REST API sources |
| `max_rows` | integer | No | `100` | Maximum rows to return, capped at the project's max row limit (default 1000). A value of 0 or below is raised to 1. |
| `project_id` | integer | No | None | Specify project if needed |

*Either `datasource_name` or `datasource_id` is required.

`query` checks every column against the project's catalog and refuses one it does not know. A data source with no
extracted metadata has no catalog to check against, so `query` refuses it with *This data source has no extracted
metadata (it was never scanned, metadata loading is off, or the scan found no tables); query it through MCP once it has
metadata.* (audited and signalled like other refusals): turn metadata loading on and refresh its metadata first. Host data sources keep their own allow-list; `dry_run` reports a missing catalog as an advisory and still
validates the rest. CloudWatch sources cannot be queried through `query` (they run Logs Insights, not SQL); Beacon's own
CloudWatch features use the configured log groups only, so restrict the IAM role to them.

**For REST API data sources**, pass a JSON query definition:
```json
{
  "method": "GET",
  "path": "/api/users",
  "parameters": { "limit": 10 },
  "resultMapping": { ... }
}
```

### `get_query_context`

Get the grounding context Beacon assembles internally for the `ask` tool, scoped to your question, so **you** (or your agent) can write well-grounded SQL instead of delegating generation to Beacon's LLM. The context contains M-Schema table renderings *with real sample values*, join paths (verified foreign keys and inferred relationships kept apart), coverage notes when the table neighbourhood was capped, human-verified golden query examples, learned patterns from usage, and matching business-glossary terms.

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `question` | string | **Yes** | None | The question you plan to answer with SQL. The context is retrieved and ranked against it |
| `datasource_name` | string | No* | None | Name of the data source to ground against |
| `datasource_id` | integer | No* | None | ID of the data source (alternative to name) |
| `project_id` | integer | No | None | Specify project if needed |
| `max_chars` | integer | No | `12000` | Maximum characters of context returned (min 1000, max 30000) |

*If the project has exactly one data source it is auto-selected; with several, pass `datasource_name` or `datasource_id` (the error lists the candidates as `id: name` pairs).

`get_query_context` needs Read in the project. Its value matches come from live probes of the data source only for a caller who may also query the project (Editor and above); a Viewer gets the matches the stored sample values already hold, and no probe runs for it.

The context shows sample values and value matches only while sample-value collection and value grounding are on for the project and every project sharing the data source (both off by default), and never under the deployment lock `Beacon:Mcp:ForbidRowDataToLlm`; they pass the same name and value screens as everywhere else (see [Sample values](https://moberghr.github.io/beacon/features/data-sources/#sample-values)). Unlike the REST metadata endpoint, which returns stored sample values to an Admin only, `get_query_context` returns them to any caller with Read in the project, since they are what grounds its SQL. A response that carries them is recorded in the row-data egress log (event `7123`, "returned to the MCP client"; see [What reaches the LLM provider](#what-reaches-the-llm-provider)).

The response opens with a header naming the data source and SQL dialect, followed by the grounding context. Sections marked **(authoritative)** are human-verified. When the context exceeds `max_chars` it is cut at the last complete section and ends with an explicit truncation note. `structuredContent` carries `data_source_id`, `data_source`, `dialect`, and `truncated`.

**Intended workflow:** `get_query_context` → write your own SQL → `dry_run` to validate → `query` to execute. Use this instead of `ask` when your agent wants full control over the SQL it runs.

### `dry_run`

Validate a SQL query through all of Beacon's safety gates **without executing it**. Use before `query` to catch problems for free.

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `datasource_name` | string | No* | Name of the data source to validate against |
| `datasource_id` | integer | No* | ID of the data source (alternative to name) |
| `sql` | string | **Yes** | The SQL query to validate (SELECT only) |
| `project_id` | integer | No | Specify project if needed |

*Either `datasource_name` or `datasource_id` is required.

**The gates**, in order: the `gate` value of each issue:

1. **`read_only`**: the shared execution gate: the regex guardrail, the dialect-aware AST validator (DML/DDL, stacked queries, comment-hidden writes), the cross-database and system catalog rules and the row limit, plus PII detection (it reports the columns that would be masked). The issue's `code` names the layer that refused the SQL, for example `guardrail`, `ast`, `cross_database` or `system_catalog`.
2. **`schema`**: the schema-catalog column check, which catches hallucinated tables and columns without a database round-trip. A data source without extracted metadata gets an advisory issue and an invalid verdict.
3. **`provider_dry_run`**: the database's own validation (`EXPLAIN` / `sp_describe_first_result_set`); it runs only when the earlier gates found nothing to fix (a missing schema catalog alone does not stop it). An engine with no dry-run strategy reports the gate as skipped with an advisory issue, and the verdict is invalid: a validation that never ran is never counted as a pass.

Issues are collected rather than first-failure-wins, so one call reports everything to fix. The response gives a per-gate verdict and, when valid, the exact SQL that would execute with the row limit applied. `structuredContent` carries `{ valid, issues: [{ gate, code, error }], executable_sql, pii_columns }`. An invalid verdict is still a successful tool call; `isError` is reserved for resolution failures.

### `get_documentation`

Retrieve AI-generated documentation at three levels of detail.

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `project_id` | integer | No | Specify project if needed |
| `datasource_name` | string | No | Get docs for a specific data source |
| `table_name` | string | No | Get detailed docs for a specific table or API endpoint |
| `schema_name` | string | No | Schema name or API tag (optional qualifier for table_name) |
| `response_format` | string | No | (any response is also cut at the project's `MaxResultBytes` with a truncation note) `concise` (summary sections) or `detailed` (everything). Project level defaults to `concise`; data-source, table and document level default to `detailed` |
| `document` | string | No | Path (or exact title) of a [host-imported document](https://moberghr.github.io/beacon/mcp/host-docs/); returns it in full |

**Three levels:**

1. **Project level** (no parameters): Full generated project documentation. Defaults to `concise`: the export is cut at ~8,000 characters on a line boundary with an explicit truncation note; pass `response_format: "detailed"` for the full document
2. **Data source level** (`datasource_name` only): Tables, schemas, code references, quality scores. Defaults to `detailed`; `concise` omits the LLM schema context section
3. **Table level** (`table_name`): Columns with types, relationships, quality rules; `detailed` (the default) adds code references and lineage, `concise` omits them

Table documentation, code references and lineage come from the resolved project only: on a data source several projects share, another project's documentation and repositories never appear.

The quality rules on table level are those of the table's enabled data contracts, each as its contract's latest evaluation found it: a disabled rule, a Custom SQL rule waiting for approval and a rule added since are not listed. Each result is shown as a status Beacon writes itself, never the result's stored text: the rule's outcome in words (`3 duplicate values found`, `Data is stale`, `The rule did not complete`) with, for a failed built-in rule, the number its actual value starts with. A Custom SQL rule shows only `Custom check passed` or `Custom check failed`: its `message` and `actual_value` are whatever its query returned, and never reach the model.

When the host ships documents with [`ExposeDocs`](https://moberghr.github.io/beacon/mcp/host-docs/), the project-level response ends with an *Imported documents* listing (title + path) and `document: "<path>"` returns one of them.

### `search`

Search tables, columns, and documentation across all data sources in the project. The keyword arm matches the words of the query, not the query as one string: "how do I request annual leave" looks for `request`, `annual` and `leave` (function words and single characters are dropped, at most 8 words, and words keep their underscores, so `order_items` stays one), and a result holding more of the words ranks higher. Keyword matching is fused with embedding-based semantic matching (reciprocal rank fusion) when a [local embedder](https://moberghr.github.io/beacon/mcp/knowledge-base/#local-embeddings) is configured; without one, search falls back to keyword-only rather than failing.

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `query` | string | **Yes** | None | Search keyword (e.g., "customer", "order_date", "revenue") |
| `project_id` | integer | No | None | Specify project if needed |
| `max_results` | integer | No | `20` | Maximum results to return (max: 50) |
| `offset` | integer | No | `0` | Result offset for paging, max `200`. Use it with `max_results` to page through large result sets |

Each result line includes the item type (`[TABLE]`, `[COLUMN]`, `[DOC]`, or `[IMPORTED_DOC]` with the document path for [host-imported documents](https://moberghr.github.io/beacon/mcp/host-docs/)), the data source and schema-qualified table (plus the column name for column hits), and the description, ordered by relevance. When more matches exist beyond the current page, the response ends with an explicit note giving the `offset` to request next.

### `feedback`

Record whether a previous `ask` answer was correct. The verdict is recorded on the answer's signal with the user who recorded it, and it does this:

- An `incorrect` verdict keeps the answer out of the common-query and join patterns Beacon learns.
- A `correct` verdict puts the answer on the [golden candidates](https://moberghr.github.io/beacon/mcp/knowledge-base/#how-cases-are-created) list (the **Golden candidates** tab of the MCP Learning page), where an Admin may promote it to a verified example (golden pair) that grounds future SQL generation for the same data source; feedback never promotes it by itself. A `correct` verdict counts in [pattern mining](https://moberghr.github.io/beacon/mcp/knowledge-base/#from-signal-to-injected-lesson) only when a curator recorded it from an interactive session (a project admin member, or an Admin, signed in); a verdict through this tool, an MCP caller or an API key, is attributed to the user it acts for but never counts as a curator's, whatever that user's roles, and neither does the asker's own.

The tool's description, its answer, the server instructions and the playground's feedback form all say so.

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `signal_id` | integer | **Yes** | The `signal_id` from the `ask` response you are rating |
| `verdict` | string | **Yes** | `correct` or `incorrect` |
| `corrected_sql` | string | No | The corrected SQL, if you fixed it |
| `note` | string | No | A short note, **only when the user explicitly asked to record one**, never a summary of the conversation |

Every `ask` response that ran a data query ends with a `_signal_id: N_` marker. Pass that value back here. Conceptual answers from the knowledge base do not record a query signal and carry no marker.

A signal is rated only in its own project: the signal must belong to a project, that project must be the one the call resolves to (in the playground, the selected project; on the transport, one of the caller's projects), and the caller needs Query there. Anything else answers "Query signal N not found.", the same as a signal that does not exist, and is audited. Within its project, a verdict is recorded only by the caller who asked, a project admin of that project, or an Admin; anyone else gets an error saying so, and the refusal is audited too. `POST /beacon/api/eval/feedback` is Admin-only: the whole `/beacon/api/eval` group requires an Admin.

`corrected_sql` is at most 100,000 characters and `note` at most 2,000; a longer one is refused and nothing is recorded. The verdict is always recorded, with the user who recorded it. Whether `corrected_sql` and `note` are *stored* depends on the project's **Allow the feedback tool to store question and SQL text** setting. See [Content retention](#content-retention) below. When that setting is off, the verdict still lands and the rest is dropped.

`corrected_sql` is checked before it is stored, in the dialect of the data source the rated answer ran on: its comments are removed, and it must pass the same read-only gate and catalog check a saved query step passes (a single read-only statement, columns that exist in Beacon's metadata; a data source Beacon holds no metadata for refuses it). Nothing runs. Comment characters inside a string literal are kept when the dialect reads them as part of it; a comment that cannot be removed safely, or characters a reviewer cannot see, refuse the SQL (see [the grounding SQL check](https://moberghr.github.io/beacon/mcp/knowledge-base/#who-curates-grounding)). Corrected SQL that cannot be kept never refuses the call: the verdict is recorded and only the SQL is dropped. This covers SQL the check refuses, SQL on a signal without a single data source (a cross-source or knowledge answer: there is no dialect to check it in), and SQL the project's retention setting does not keep. The log records the signal id only.

**The answer** starts with "Feedback recorded for signal N.", says what the verdict does, and, when `corrected_sql` was given, whether it was kept: "The corrected SQL was kept with the verdict, without its comments.", or "The corrected SQL was not kept:" with the reason (the project does not retain query content; the answer did not run on a single data source; it did not pass the SQL check, with the check's reason). `POST /beacon/api/eval/feedback` answers the same as `correctedSqlRecorded` (true when corrected SQL was given and kept) and `correctedSqlNotRecordedReason` (null when it was kept or none was given); the playground's feedback form shows the reason.

## Safety & guardrails

The MCP server enforces several safety measures:

| Feature | Description | Default |
|---------|-------------|---------|
| **Read-only enforcement** | Only read-only statements run | On; locked on by default (`Beacon:Mcp:ForceReadOnly`) |
| **Database names** | SQL may name only its data source's configured database (catalog, project); other qualifiers are refused | Always on |
| **Database boundary** | Functions and procedures that reach other servers, files or storage, SQL-text extension functions, PostgreSQL `COPY … PROGRAM` / server-file `COPY`, Databricks file paths and four-part names are refused | Always on |
| **Shared data sources** | A source several live projects link runs, and is grounded, with the strictest guardrails among them | Always on |
| **Row limits** | `query`: `max_rows` (default 100) capped at the max row limit; `ask` on one data source: 100 rows, or the max row limit when that is lower; cross-source: the max row limit | Max row limit 1000 |
| **PII detection** | Detects and masks sensitive values in results | On; locked on by default (`Beacon:Mcp:ForcePiiDetection`) |
| **Query timeout** | Each statement `query`, `ask` and the saved-query tools run may take the project's **Statement timeout** (other Beacon paths use `Beacon:Query:StatementTimeoutSeconds`, default 120) | 30 seconds |
| **Plan cost** | On a PostgreSQL data source, each statement `query`, `ask` and the saved-query tools run is first planned with `EXPLAIN (FORMAT JSON)` (never `ANALYZE`; the same read-only, single-statement path and statement timeout as the statement, with its parameters) and refused before it runs when the planner's estimated total cost exceeds the project's **Max EXPLAIN cost**, with a message naming the estimate and the limit; the refusal is audited. An estimate that fails fails the call as the statement would. Other engines ignore the setting | No limit |
| **Result size** | The text and the structured result of `query`, `ask` and the saved-query tools are held together to the project's **Max result size**: the structured result takes at most half, the text (heading, markdown table and notes) the rest. A larger result returns fewer rows, marked as truncated by size (`truncated` plus `rows_omitted_for_size`, `rowsOmittedForSize` on saved-query tools, and a note in the text). Not the deployment's `Beacon:Query:MaxResultBytes` (256 MiB), the budget a result is read under from the data source, past which the run fails (see [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/#query-execution-row-cap)) | 262,144 bytes |
| **Concurrent queries** | One credential (an API key, else a mapped caller, else a Beacon user) may run up to the project's **Max concurrent queries per API key** `query`, `ask`, `dry_run` and saved-query calls at once on each Beacon instance; another call is refused at once with *Too many concurrent queries for this credential; retry shortly.* and audited | 4 |
| **Request rate** | One API key or mapped caller may send `Beacon:RateLimits:Mcp` requests a minute to `/beacon/mcp` on each Beacon instance; more answer HTTP `429` (`application/problem+json`, `Retry-After`) before the MCP server reads them. `ask` is also limited per caller (`Beacon:RateLimits:Ask`, MCP and playground together); another question is refused with *Too many questions for this credential; retry in N seconds.* and audited. See [Request limits](https://moberghr.github.io/beacon/getting-started/configuration/#request-limits) | 60 requests, 10 questions a minute |
| **Audit logging** | Every tool call is recorded by `McpAuditService` with user, timing, and parameters (see [MCP observability and audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/)) | Always on |
| **Usage signals** | `ask`, `query`, and `dry_run` calls are recorded by `McpSignalService` to feed the learning loop | Always on |
| **Content retention** | Whether questions, SQL, and error text are persisted at all, per project, lockable deployment-wide | Retained |

**Database names.** `ask`, `query`, `dry_run` and the saved-query tools refuse SQL that names another database than the data source's own (`database.schema.table` on SQL Server, Azure Synapse, PostgreSQL and Snowflake, `database.table` on MySQL, `catalog.schema.table` on Databricks, `project.dataset.table` on BigQuery) with the gate code `cross_database`, also when read-only enforcement is off. The allowed database is the one the data source's connection configures; a source without one, or whose connection settings cannot be read, accepts no database qualifier. Unqualified and schema-qualified names always pass. The same always-on stage refuses `dblink…`, `lo_…`, `xp_…`, `OPENROWSET`, `OPENQUERY`, `OPENDATASOURCE`, `EXTERNAL_QUERY` and the other functions that reach other servers, files or storage (on Databricks also `http_request`, `read_kafka` and the other stream readers, `reflect` and `java_method`), the PostgreSQL extension functions that run SQL text (`pg_background_…`, pgRouting's `pgr_…`), PostgreSQL `COPY … TO/FROM PROGRAM` and `COPY` to or from a server file, Databricks file paths and SQL Server four-part names, so turning read-only enforcement off never opens them. See [Database names in SQL](https://moberghr.github.io/beacon/security/authorization/#database-names-in-sql).

**System catalog.** The same tools refuse SQL that reads the engine's catalog, statistics or query history (`information_schema`, `pg_catalog` and `pg_…` relations, `sys`, Snowflake `QUERY_HISTORY` and `RESULT_SCAN`, BigQuery `INFORMATION_SCHEMA`, …) with the gate code `system_catalog`, also when read-only enforcement is off. `get_context` and `get_query_context` describe a source's tables and columns from Beacon's metadata.

**Shared data sources.** The guardrails a tool applies are the data source's: when several live projects link it, read-only enforcement and PII detection are on if any of them turns them on, value grounding and sample values are off if any of them turns them off, all their custom PII patterns apply, and the lowest row limit and value-grounding probe budget win, whichever project the caller works in. The execution settings (statement timeout, result size, plan-cost limit) resolve to the lowest value the same way. `ask` applies them over every data source it routes to, and grounding (`ask`, `get_query_context`) applies the value-grounding, sample-value and PII settings. A metadata refresh, which no project asks for, collects sample values while collection is on in the effective settings of every live project that links the source (each project's override, else the global value); the global value alone decides only for a source no live project links. See [Guardrails of shared data sources](https://moberghr.github.io/beacon/security/authorization/#guardrails-of-shared-data-sources).

| Setting | Strictest among the linking projects |
|---------|--------------------------------------|
| `enforceReadOnly`, `enablePiiDetection` | on when any project turns it on |
| `enableValueGrounding`, `enableSampleValueCollection` | off when any project turns it off |
| `customPiiPatterns` | every project's patterns |
| `maxRowLimit`, `statementTimeoutSeconds`, `maxResultBytes`, `maxExplainCost`, `valueGroundingMaxProbes` | the lowest (`maxExplainCost`: any limit over none) |

`McpAuditService` fires on every tool invocation, including the failure path, and it is never short-circuited. Audit rows and usage signals record the caller by its Beacon user (`Users.Id`): an API key's owner, a mapped caller's provisioned user, or the signed-in user in the playground, never the session's own identifier, so a caller can rate the signals of its own `ask` calls. A tool call refused for a missing `Execute` scope (or an unauthenticated caller) is audited too, with the tool name, the caller's identifiers and a permission error; when `Beacon:Mcp:Audit:Required` is set and that row cannot be written, the refusal answers with the withheld-result message instead. `McpSignalService` records the three SQL-carrying tools (`ask`, `query`, `dry_run`); the catalog tools (`get_context`, `search`, `get_documentation`, `get_query_context`) are audit-only by design.

## Content retention

By default Beacon stores what an agent asked and what SQL ran. That is what makes the learning loop and the audit trail useful. For a project handling regulated or sensitive data you can turn that off without losing the audit trail itself.

Switch **Retain question and SQL text in signals and audit** off for a project and Beacon keeps the **structure** of every interaction and stops persisting its **content**:

| Kept | Dropped |
|---|---|
| Tool, project, data source, user, caller | The natural-language question |
| Tables and columns referenced | Generated, corrected, and user-supplied SQL |
| Row counts, timings, success flags, verdicts | Routing rationale and feedback notes |
| An **error class** (see below) | Free-text database error messages |
| The audit row itself: always | The audit row's raw parameters |
| Keyed hashes of the parameters and of each statement run | The text of each statement the call ran |

An audit row is still written for **every** tool call, success or failure. The lock changes what the row contains, never whether it exists.

While content is retained, the audit keeps the SQL of **every statement** a call ran (an `ask`'s first execution, repairs, self-consistency candidates, value-grounding probes and dry runs) beside the question; see [The statements a call ran](https://moberghr.github.io/beacon/mcp/observability-and-audit/#the-statements-a-call-ran). A deployment that must keep this evidence pins retention on with `Beacon:Mcp:Audit:RequireContent` (see [Keeping audit content](https://moberghr.github.io/beacon/mcp/observability-and-audit/#keeping-audit-content-the-requirecontent-lock)).

**Error classes.** Instead of the provider's message, Beacon stores one of nine values: `schema`, `syntax`, `permission`, `timeout`, `not_found`, `validation`, `execution`, `cancelled`, `unknown`. A tool's own argument errors (an invalid, unknown or missing argument) and a host endpoint's `400` or `422` are `validation`, whatever the argument is called. You keep the ability to chart failure modes without keeping text that may quote customer data.

**Audit parameters** become a structural summary: which tool ran, how many bytes of input it received, and which tables it touched:

```json
{ "tool": "query", "params": [{ "name": "input", "bytes": 412 }], "tables": ["orders", "customers"] }
```

**What else changes under the lock**

- The `feedback` tool records the verdict but not the corrected SQL or note. Signals of a locked project carry no question, so they are not offered as golden candidates.
- Pattern mining keeps schema corrections and join patterns (structure) and skips common-query and documentation-gap mining (content). Join patterns are stored without example questions or SQL, and the LLM lesson extractor is not called, because its prompt would carry the question and SQL.
- The optional eval judge is forced off, and eval results are stored without generated SQL or judge commentary.

**Enforcement.** Every write path applies the rule, and an EF Core `SaveChangesInterceptor` backs them up: content is stripped on the way to the database even if a future code path forgets, and it fails closed: if the settings cannot be read, the row is treated as locked. All of this is logged with identifiers only.

Retention is **not retro-active**: rows written before you turned the lock on keep their content until they age out of the retention window. A row that is *updated* after the lock is on is cleaned up by the interceptor.

**Outside MCP.** Turned off globally (or pinned off with `Beacon:Mcp:ForceNoContentRetention`), the same rule covers Beacon's other content stores: the SQL and stored result snapshot of a subscription run and its notifications, the statement text of the execution log (its error text becomes an error class), the analysis, findings and conversation of an AI actor, and the statement of a migration run. A project override does not reach them: they belong to no single project. The [data retention job](https://moberghr.github.io/beacon/getting-started/configuration/#data-retention-beaconretention) clears result snapshots stored before the lock, and with `Beacon:Retention:ContentDays` the content of older rows.

## SQL accuracy stack

Natural-language questions through the `ask` tool don't go through a naive prompt-to-SQL pipe. Every generated query passes a layered accuracy stack:

1. **Grounded context**: before generation, Beacon assembles an M-Schema rendering (column name, type, nullability, description), *with real sample values and value matches* when a project turns sample-value collection and value grounding on (both off by default; see [What reaches the LLM provider](#what-reaches-the-llm-provider)), so filters match actual data formats (`'shipped'` vs `'SHIPPED'`), plus curated join paths, matching business-glossary terms, human-verified query examples, and lessons mined from past usage. See the [Knowledge Base guide](https://moberghr.github.io/beacon/mcp/knowledge-base/) for what goes in and how you curate it.
2. **AST read-only validation**: generated SQL is parsed into an abstract syntax tree in the data source's own dialect (PostgreSQL, SQL Server, MySQL, BigQuery, Snowflake, Databricks); SQL for a source whose dialect is unknown is refused. DML/DDL statements, stacked queries, comment-hidden writes, row-locking clauses, functions that read files, reach other servers or sleep, linked-server (four-part) names and file path references are rejected before anything reaches your database. This is defense-in-depth beyond the regex guardrail.
3. **Database read-only backstop**: on PostgreSQL and MySQL, MCP-executed SQL additionally runs inside a `READ ONLY` transaction (on PostgreSQL as a batch of exactly one command, which the driver and the server refuse to split into several statements; on MySQL also with a read-only session default and statement batches turned off), so a write that somehow slipped past both parsers is still refused by the database itself. This is defence in depth, not a replacement for a read-only login. BigQuery and Databricks have no read-only session: their connectors refuse, on the read-only path, anything their own dialect's parser does not read as a single `SELECT` in the configured project or catalog, and BigQuery additionally runs only what a dry run reports as a `SELECT`. SQL Server, Azure Synapse and Snowflake rely on the parser gates (SQL Server also runs the statement in a transaction that is always rolled back, and fails the run when the statement ends that transaction itself). On all five, the read-only login you connect with is the database-level boundary; on PostgreSQL and MySQL it is still required.

The same gate and read-only path run every query Beacon executes outside MCP too: previews, subscriptions, data-quality rules, AI actors, migration reads; see [How a query runs](https://moberghr.github.io/beacon/features/queries/#how-a-query-runs).
4. **Dry-run repair loop**: if execution fails, the SQL is retried with the database error and a refreshed schema context; the error goes into the prompt redacted by its class (see [Database errors](#database-errors)); truncated or degenerate repairs are rejected rather than executed.
5. **Row limits & PII masking**: results are capped and sensitive values (emails, phone numbers, SSNs, credit cards, tokens, and custom regex patterns) are masked (`a***z`) before leaving the server. A column is masked when its own name is PII, when it is an alias over a PII column, or when it reads a JSON document or map by a literal PII key (`profile ->> 'email'`, `profile #> ARRAY['email']`, `JSON_VALUE(profile, '$.email')`, Snowflake `v:email`, `m['email']`). The masker cannot parse a Databricks `v:email` path, so a statement that runs with one has every result column masked, as has a SQL Server `FOR JSON` or `FOR XML` statement, which folds every row into one column. A key column named after a contact word (`address_id`, `zip_code_id`) holds an id and is not masked; an `EXISTS` or `IN (subquery)` test is a boolean and is not masked either. A host data source's masked columns come back as `***`, masked once and in full; when the host policy check does not pass, every result column comes back as `***`.

## Learning loop

The MCP server is self-improving. `McpSignalService` records a usage signal for every SQL-carrying call, holding the question, the generated SQL, the tables and columns referenced, the routing decision, and the outcome. Recurring background jobs turn those signals into knowledge that grounds future generation:

![MCP Learning](https://moberghr.github.io/beacon/img/screenshots/mcp-learning-dark.png)

| Job | Schedule | What it does |
|---|---|---|
| `mcp-learning-aggregate` | Every 6 hours | Mines `ask` / `query` signals into learned patterns, retires stale ones, and runs replay verification |
| `mcp-learning-cleanup` | Daily 03:00 | Drops signals past the retention window |
| `mcp-embedding-reindex` | Every 12 hours | Re-embeds schema metadata and exemplars |
| `mcp-docchunk-reindex` | Every 12 hours | Re-chunks and re-embeds project documentation |

Two things are worth knowing about how patterns get promoted:

- A candidate pattern is not trusted on a confidence score alone. Before promotion, Beacon **replays the project's golden cases** with and without the candidate injected, and keeps it only if it flips at least one case from failing to passing.
- `dry_run` signals are recorded for analytics but **excluded from mining**: their "question" is SQL, not natural language, and gate rejections would skew the failure statistics.

Review the queue on the **MCP Learning** page (`/beacon/mcp-learning`), where pending patterns can be approved or rejected, and any stored documentation patch applied or rejected. The full mechanics (lesson extraction, decay, retention, and every setting) are in the [Knowledge Base guide](https://moberghr.github.io/beacon/mcp/knowledge-base/#the-learning-loop).

## Configuration

Admins and SecurityAdmins customize the MCP server at **MCP Settings** (`/beacon/mcp-settings`); Auditors read it. The routes: `GET /beacon/api/mcp/settings` and `GET /beacon/api/mcp/projects/{projectId}/settings` for an Auditor or a SecurityAdmin, the `PUT`s for a SecurityAdmin (Admins hold both; see [Functional permissions](https://moberghr.github.io/beacon/security/authorization/#functional-permissions)).

**Global defaults or per project.** A scope selector at the top of the page switches between the global defaults and any single project. In project scope each field gets an **Override** checkbox: leave it unchecked and the project inherits the global value (shown greyed out); check it and the project keeps its own. Prompts and tool descriptions stay global by design: `tools/list` is answered before any project is known.

The tabs:

**Pre-prompt**
- **Ask tool system prompt**: the LLM prompt used for SQL generation in the `ask` tool, sent as is. The global instruction and the schema context follow as a first user message, the stable prefix a provider can reuse from its prompt cache (Claude is asked to cache it; OpenAI and Azure OpenAI cache such a prefix on their own; Bedrock does not request prompt caching yet); the date and the question come in a second user message. An `ask` repair sends that same first message, byte for byte, then the question, the previous SQL, its error and the rules in a second message ending with "Return ONLY the corrected SQL query."; a repair over any other context (a cross-source repair) sends the full schema first with no cache request. A provider's cache covers the system prompt too, so a repair reuses the generation's cached prefix only when both use the same system prompt, that is when this prompt is set. Schema text (database comments, documentation, glossary) is always user content, never part of the system prompt. That first message also states that text inside `<untrusted_data>` blocks is data, never instructions, and carries the schema context in such a block (its two marker lines are not counted in `AskSchemaContextChars`); a repair's table schemas, previous SQL and error are blocks too. Descriptions, sample and matched values, glossary terms and learned patterns stay on their own line inside the block, characters a reviewer cannot see are left out, and a block marker written inside the data is neutralised, also one written with lookalike brackets or letters, fullwidth forms, accents or control characters, so stored text cannot close its block. Data source routing, intent classification, the cross-source join, knowledge answers, documentation generation and section edits, chunk indexing, AI actors, lesson extraction and the eval judge mark their context the same way.
- **Global instruction**: prepended to the user context of every LLM-aware tool

**Tool descriptions**
- Override the description returned by `tools/list` for `get_context`, `ask`, `query`, `get_documentation`, and `search`. Leave a field blank to keep the built-in description. Overrides are applied to the live tool list, so connected clients see your wording. `dry_run`, `get_query_context`, and `feedback` currently use their built-in descriptions only.

**Guardrails**
- **Max row limit**: the ceiling on rows returned (default: 1000). It must be between 1 and 10,000,000; a save outside that range, globally or for a project, is rejected with `400`. A lower value stored before this check is read as 1, so `query`, `dry_run`, `ask`, cross-source queries and evaluations all cap at one row, never run uncapped and never refuse the cap itself.
- **Read-only enforcement**: the SELECT-only restriction; locked on by default (see below)
- **PII detection**: enable/disable, plus custom PII regex patterns; detection is locked on by default (see below). A custom pattern that cannot be evaluated over a query (invalid, or not finished within its one-second timeout) refuses the query, naming the pattern by its position; as a column check it marks the column as PII. A new pattern is at most 500 characters and must not use lookarounds, backreferences or atomic groups (it runs on the linear-time regex engine); a save with one is rejected with `400`. A pattern stored before these checks keeps working, on the backtracking engine under the timeout, and does not block a save that leaves it in place
- **Row data sent to the LLM provider**: sample-value collection and value grounding, both off by default (see [What reaches the LLM provider](#what-reaches-the-llm-provider))
- **Execution**: the two retention switches, **Retain question and SQL text in signals and audit** and **Allow the feedback tool to store question and SQL text** (see [Content retention](#content-retention)); and **Statement timeout (s)** (default 30), **Max result size (bytes)** (default 262144), **Max EXPLAIN cost** (blank = no limit) and **Max concurrent queries per API key** (default 4). The statement timeout, result size and concurrent-query limit apply to the MCP SQL tools (see the guardrails table above); the EXPLAIN cost limit is enforced on PostgreSQL data sources, where a statement whose planner-estimated total cost exceeds it is refused before it runs, and other engines ignore it
- **Learning**: **Enable learning**, auto-approve threshold (stored; auto-approval is decided by replay), injection budget, and signal retention

**Ranges.** Every numeric setting is checked when it is saved, globally or as a project override (an empty override inherits and is not checked); a value outside its range is rejected with `400` and a message naming the field and the range. A save names the first field out of range; fix it first, then save again to see the next. A deployment ceiling is not a bound here: a value above it is saved and clamped when the settings are read.

| Setting | Allowed |
|---------|---------|
| `maxRowLimit` | 1–10,000,000 |
| `statementTimeoutSeconds` | 1–3600 |
| `learningSignalRetentionDays` | 1–36,500 |
| `learningAutoApproveThreshold` | 0–1 |
| `maxExplainCost` | blank (no limit), or 0–9,999,999,999,999,999.99 |
| `maxResultBytes`, `maxConcurrentQueriesPerKey`, `selfConsistencyCandidateCount`, `selfConsistencyMinTables`, `exemplarTopK`, `glossaryTopK`, `docChunkTopK`, `goldenExemplarTopK`, `docChunkWindowSentences`, `learningReplayMinFlips` | at least 1 |
| `learningInjectionBudgetChars`, `goldenExemplarBudgetChars`, `valueGroundingMaxProbes`, `docChunkOverlapSentences` | at least 0 (0 = none) |

A statement timeout, result size or concurrency limit of 0 or less stored before these checks is read as its default (30 seconds, 262,144 bytes, 4) globally, and as no override (the global value) for a project; a statement timeout above 3600 seconds is read as 3600. Each such field is reported as clamped. The **Max result size** here bounds what an MCP tool returns; the deployment's `Beacon:Query:MaxResultBytes` bounds what is read from the data source.

**Context preview**
- Render the grounding context for a project exactly as the tools would assemble it. This is useful for spotting missing documentation before an agent hits it.

### Deployment locks and ceilings

Some settings must be non-negotiable no matter what an administrator clicks. The `Beacon:Mcp` configuration section pins values at the host level. Read-only enforcement and PII detection are pinned **by default**: `ForceReadOnly` and `ForcePiiDetection` are `true` unless the host's configuration sets them to `false`.

```json
{
  "Beacon": {
    "Mcp": {
      "ForceReadOnly": true,
      "ForcePiiDetection": true,
      "ForceNoContentRetention": true,
      "ForbidRowDataToLlm": true,
      "Ceilings": {
        "MaxRowLimit": 1000,
        "StatementTimeoutSeconds": 60,
        "MaxResultBytes": 1048576,
        "MaxConcurrentQueriesPerKey": 8,
        "MaxSqlChars": 100000,
        "MaxQuestionChars": 4000,
        "AskTimeoutSeconds": 120,
        "AskMaxLlmCalls": 12,
        "AskSchemaContextChars": 120000
      }
    }
  }
}
```

- A **lock** (`ForceReadOnly` and `ForcePiiDetection`, both on by default; `ForceNoContentRetention`, `ForbidRowDataToLlm` and `Audit:RequireContent`, off by default) pins the effective value for the global settings and every project. `Audit:RequireContent` pins content retention on, the opposite of `ForceNoContentRetention` (the host does not start with both on). `ForbidRowDataToLlm` pins three settings off: sample-value collection, value grounding and the eval judge (see [What reaches the LLM provider](#what-reaches-the-llm-provider)). Nobody can change a locked setting at runtime, an Admin included; only the host's configuration lifts the lock. The global page shows a locked switch disabled with a note, the project scope hides locked fields and names them in a banner, `GET /beacon/api/mcp/settings/locks` lists them (`lockedFields`, and each lock with the setting it pins in `locks`), and an API call that tries to set one gets **HTTP 409** with the RFC 7807 type `/errors/setting-locked`.
- With `ForceReadOnly` on, the SQL execution gate also judges every statement read-only whatever the settings a caller hands it. This covers the `ask`, `query` and `dry_run` tools and the in-memory join of a cross-source query. Only a deployment that sets `ForceReadOnly: false` lets an Admin turn read-only enforcement off (globally or per project), and only there does the setting reach the gate. The host logs a warning at startup for each of these two locks that is off.
- A **ceiling** clamps a numeric value downward only. It can lower what an administrator configured, never raise it. Clamped fields are flagged in the UI. `MaxRowLimit` bounds the rows the tools return; `StatementTimeoutSeconds`, `MaxResultBytes`, `MaxExplainCost` and `MaxConcurrentQueriesPerKey` clamp the settings of the same names.
- `Ceilings:MaxSqlChars` (default `100000`) and `Ceilings:MaxQuestionChars` (default `4000`) cap the length of SQL and of `ask` questions, and `Ceilings:AskTimeoutSeconds` (default `120`, 1 to 3600) and `Ceilings:AskMaxLlmCalls` (default `12`, at least 2: the intent classification and the first SQL generation) cap the wall-clock time and the LLM calls of one `ask` call (see [`ask`](#ask)), and `Ceilings:AskSchemaContextChars` (default `120000`, about 30,000 tokens) caps the schema context one `ask` sends to the LLM: the first generation, every self-consistency candidate and every repair get the same bounded context. Over the cap, whole table descriptions are left out, never cut mid-table: the tables the question names as whole words stay first (only the one in that schema when the question writes `schema.table`), then the tables retrieval ranked for the question, then the rest in catalogue order; a table whose description does not fit is skipped and a shorter one after it may still be shown. One line `-- N more tables not shown: …` names the tables left out (the list itself capped near 2,000 characters) and replaces the coverage note; it is counted inside the cap. When the grounding sections leave no room for the most relevant table they are left out one kind at a time — learned patterns, then the glossary, the verified examples, the value matches and last the join paths — until it fits, a table is never sent whole when it does not fit: if no whole table fits, the most relevant one is cut at the last line that fits and followed by `-- name: more columns not shown` (a warning with counts only is logged), so the context is never longer than the cap and never describes no table when the source has one. A repair over SQL that reads a table left out gets that table's exact schema, however the SQL spells its name (quoted, unqualified or with a database in front), in the single-source `ask` and in each source of a cross-source `ask`. The `get_query_context` tool and cross-source `ask` read the same bounded context; a cross-source `ask` bounds each source's context on its own, so the join step's prompts can carry up to the cap once per source. They **always apply** at their defaults, even when the section is absent; the length caps must be greater than 0 when set, and the context cap at least 10000. With no section, read-only enforcement and PII detection are locked on, content retention and row data to the LLM are not locked, there are no deployment-set ceilings, and these length caps and `ask` budgets hold.
- Settings resolve as **lock → project override → global value → built-in default**, then ceilings are applied.
- `GET /beacon/api/mcp/settings` returns the **resolved** settings (what the tools read, locks and ceilings applied) as before, plus `stored` (the row as saved, before locks and ceilings), `lockedFields` and `clampedFields` (the settings a ceiling lowers). The global page edits `stored`, so a save keeps a value above a ceiling and the value a lock overrides: lifting the lock or raising the ceiling restores what was configured underneath. `PUT` stores every setting as sent, except a locked one, which it keeps as stored (and refuses with `409` when the value sent contradicts the lock). An API client that sends the resolved values back stores them: send `stored` instead.
- **Every change is recorded.** Each setting a save changes, globally or for a project, is written to the settings history the [Admin Settings](https://moberghr.github.io/beacon/security/admin-settings/) page keeps: key `Mcp.{Setting}` or `Mcp.Project.{projectId}.{Setting}`, the value before and after (an empty override is "inherit"; a long prompt or description is cut to 2,000 characters), when, and who (the caller's Beacon user id). The log notes the change by setting name and ids only.

:::caution[Upgrading]
Read-only enforcement and PII detection can no longer be turned off at runtime unless the deployment unlocks them. A
deployment that had either switched off globally or for a project runs with it on after the upgrade; the stored value is
kept, and takes effect again only if the host sets `Beacon:Mcp:ForceReadOnly` (or `ForcePiiDetection`) to `false`. The
host logs one warning after startup listing the stored rows a lock overrides (the global row and project ids), and a
statement refused only because the lock turned read-only enforcement on names `Beacon:Mcp:ForceReadOnly` in its reason.
:::

### What reaches the LLM provider

Beacon sends prompts to the LLM provider configured in [Admin Settings](https://moberghr.github.io/beacon/security/admin-settings/). Besides the question,
the schema (table and column names, types, descriptions, join paths) and your curated knowledge (glossary, documentation,
verified examples, learned patterns), a prompt can carry data read from your data sources. Each kind is off by default
or screened, and each is logged:

| Row data (egress category) | When | Screened |
|---|---|---|
| Sample values (`sample-values`, `Examples: [...]`) in `ask` and repair prompts, and returned by `get_query_context` | sample-value collection on (off by default) in the effective settings of the project and of every project sharing the source | name and value rules of [Sample values](https://moberghr.github.io/beacon/features/data-sources/#sample-values), again at every read |
| Value matches (`value-matches`, `## Value matches`) for literal-looking words of the question | value grounding on (off by default), same sharing rule; live probes only for callers who may query | PII, secret-like and financial-amount column names are never probed or matched; a PII- or credential-shaped hit is dropped |
| Distinct column values in documentation prompts (`column-values`, enum discovery) | sample-value collection on for the project and every project sharing the source | the same name and value rules |
| Result rows in the eval judge's prompt (`result-rows`) | the eval judge on (off by default) and content retention on | PII columns masked, at most 20 rows |
| Database errors in repair prompts and in lesson extraction (`database-errors` for lesson extraction) | an `ask` or cross-source `ask` repair after a failed dry run or execution; the learning job mining failed `ask` signals | redacted by class ([Database errors](#database-errors)); under the lock repair prompts and lesson extraction send only the class and code |
| Stored documentation of the business-domain, data-model and glossary sections (`stored-documentation`), which can list values discovered in the data | knowledge answers: the documentation, its chunks and the project search results cut from these sections (keyword and semantic); contextual-retrieval blurbs of documentation chunks; a documentation instruction on such a section | under the lock these sections, their chunks and their search results are left out of knowledge answers and blurbs, and an instruction on one is refused (`400`); regenerate the documentation instead |

- **The deployment lock `Beacon:Mcp:ForbidRowDataToLlm`** (off by default) pins sample-value collection, value grounding
  and the eval judge off for the global settings and every project: no sample is collected or shown, no value is probed,
  documentation discovers no column values, the judge sees no rows, repair prompts and lesson extraction send only an
  error's class and code, and stored documentation sections that can list discovered values stay out of every prompt. A
  request that would turn one of the three settings on answers `409`, and a project override of one is refused even when
  it agrees with the lock; a save under the lock keeps the values stored before it (globally and per project), so
  lifting the lock restores them. After startup the host logs one warning listing the stored rows the lock overrides
  (global row and project ids), and one naming the projects whose stored documentation still has value-listing sections;
  values discovered before the lock stay in that documentation until it is regenerated (a curated section, one a
  person edited, is kept by regeneration: edit it to remove them).
- **Egress log.** Every request whose prompt carries row data is logged at Information with event id `7123`
  (`RowDataSentToLlm`) as it is handed to the provider. The data may leave even when the provider then fails, so a
  failure adds a Warning under the same event with the failure's type and code. Each record names the categories, the
  data source ids, the project id and the caller: the caller's Beacon user id, or for background work the job itself
  (`learning-aggregation`, `documentation-indexing`, `pattern-replay`; an eval run names the user who started it, else
  `eval-run`; documentation generation the user who asked for it), and `unknown` when the caller cannot be resolved
  (which never fails the request). A `get_query_context` response carrying sample values or value matches is recorded
  under the same event as returned to the MCP client. Never a value or prompt text; no table is written. See
  [MCP observability](https://moberghr.github.io/beacon/mcp/observability-and-audit/#row-data-sent-to-the-llm-provider).
- **Where the data goes.** OpenAI and Anthropic are called on their public API endpoints; there is no base-URL or proxy
  setting for them. Azure OpenAI takes your resource's endpoint (choose its region for residency) and Bedrock your AWS
  region, both in the LLM connection settings.

### Database errors

A database quotes row values back in its errors. A failed conversion names the value it could not convert, and a key
violation names the key. Beacon therefore judges an error by its **class** (SQLSTATE, error number or the engine's wording) before
it leaves its hands:

- An error that **carries values** is replaced by its class, code and a generic description, with none of its message
  kept. Such errors are a data exception or an integrity violation (SQLSTATE `22xxx`, `23xxx`), SQL Server's
  conversion, overflow, duplicate-key, truncation and NULL errors, MySQL's incorrect-value, duplicate-entry,
  out-of-range and too-long errors, and BigQuery, Databricks and Snowflake cast and parse errors. For example:
  `Database error 22P02 (data exception): a value could not be converted to integer. The database's message is withheld because it can quote row values.`
- An error that **carries no values** (an unknown or ambiguous name, a grouping, syntax, type, permission or
  division-by-zero error) keeps its message with quoted text, key and row details and unquoted values replaced by
  `<value>`. Quoted text survives only in an error about a name, and only when every part of it is a name of the
  statement or the catalog.
- An error Beacon does not recognise is replaced like a value-carrying one, whatever it says, quoted or not:
  `Database error (unrecognised): not an error Beacon recognises. The database's message is withheld because it can quote row values.`
  So is an error the statement's own code raised, whose wording is its author's: a SQLSTATE of class `P0`
  (PostgreSQL's `RAISE`, Databricks' `raise_error`), SQL Server's errors 50000 and above (`THROW`, `RAISERROR`) and
  MySQL's SQLSTATE class `45` (`SIGNAL`) keep only their code, even when the text reads like a name, syntax or
  permission error. Where the driver leaves the code out of the message, Beacon puts it in front
  (`Msg 50000, Level 16, State 1: …`, `45000: …`), also in the raw message a caller reads while PII detection is off.
- Beacon's own fixed texts that arrive with the database's errors (a host source's failure, a read stopped at the row
  ceiling, a refused read of a migration write target) hold no value and are kept. The whole message is redacted
  before it is cut to 2,000 characters; an error that cannot be screened in time is replaced as a whole and counted
  (`beacon.error_redaction.fallbacks`).

Where the redacted form is used:

| Where the error goes | Redacted |
|---|---|
| Repair prompts (`ask`, cross-source `ask`) and lesson extraction | always; under the lock `Beacon:Mcp:ForbidRowDataToLlm` only the class and code (`Database error 42703 (undefined column).`) |
| Learning signals (`ask`, `query`, `dry_run`, cross-source) and the stored history of ad-hoc queries | always |
| The error returned by `ask`, `query`, `dry_run`, cross-source queries, saved-query tools and REST previews and executions, and the audit row | while PII detection is on for the project (or the strictest project sharing the source); the raw message otherwise |
| Host data sources | never the database's message: `Query failed on the host database.` |

Logs never carry the message at all: a failed query is logged by exception type, error code, the types of the
exceptions it wraps and the method that threw it (`PostgresException (error code 22P02, inner SocketException, at NpgsqlConnector.ReadMessage)`).

:::caution[Upgrading]
- **Defaults.** A new installation starts with sample-value collection and value grounding off. Stored settings are not
  changed: an installation that saved its global MCP settings keeps the values it stored (the migration
  `DefaultSampleValueCollectionOff` only changes the column default for a row created later). An installation that never
  saved them reads the new defaults, so both are off until a project or the global settings turn them on; the host
  then logs at startup which row-data settings are in effect. With sample-value collection off, documentation
  generation lists no discovered column values (logged once per run).
- **Sampling, in both directions.** A project that turns sample-value collection on now collects even while the global
  value is off (before, the global value had to be on too). A refresh no longer reads excluded columns at all, and more
  names are excluded: person names, Croatian personal data, connection strings, `dsn`, webhook URLs, `hmac` and key
  material named with `hmac`, `aes`, `rsa`, `pgp`, `gpg` or `ssh` beside `key`. Surrogate keys such as `ProductKey` are
  no longer excluded and are sampled again. The next refresh replaces what changed.
- **Settings API.** A `PUT /beacon/api/mcp/settings` body that omits `enableSampleValueCollection` or
  `enableValueGrounding` now stores `false` (the default) for it; send every field you read. The settings page shows
  both switches and sends back every setting it loaded, the ones it does not show included.
- **Errors.** Database errors returned to callers and written to the audit are in the generic class form above while PII
  detection is on, and learning signals and ad-hoc query history always hold the redacted form. An error of no class
  Beacon recognises is now withheld even without a quote in it, and an error raised by the statement's own code
  (SQL Server 50000 and above, MySQL SQLSTATE class 45) reaches a caller led by its code. Failure logs name the
  failure as `Type (error code …, inner …, at …)` instead of separate type and code fields.
- **Masking.** Banking identifiers (`iban`, `oib`, `jmbg` as whole name segments, `account_number`) are PII names: query
  results mask such columns on every path while PII detection is on. Data-quality custom SQL masks a text value (an
  `actual_value` or `message`) that looks like PII, whatever its column is called.
- **Under the lock**, an instruction on a business-domain, data-model or glossary documentation section answers `400`.
:::

The per-project settings have their own endpoints: `GET /beacon/api/mcp/projects/{projectId}/settings` (Auditor or SecurityAdmin) and `PUT` on the same route (SecurityAdmin). The `GET` returns the project's overrides, the resolved effective values, and the names of any locked or clamped fields.

Changes take effect without a restart. These settings are part of [Admin Settings](https://moberghr.github.io/beacon/security/admin-settings/); the retrieval, exemplar, and eval knobs that aren't on this page are settable through `PUT /beacon/api/mcp/settings` and documented in the [Knowledge Base guide](https://moberghr.github.io/beacon/mcp/knowledge-base/#settings-reference).

## MCP Playground

You do not need to wire up an external MCP client to try the server. The UI ships an **MCP Playground** (`/beacon/mcp-playground`) where you select a project and ask questions interactively. It uses the same tools, routing, guardrails, and audit trail as a real MCP session. Use it to validate documentation quality and tune the system prompt before pointing Claude or Cursor at your data.

The playground runs as the signed-in user and checks their [project access](https://moberghr.github.io/beacon/security/authorization/#project-access) on every call: the catalog tools (`get_context`, `get_documentation`, `search`, `get_query_context`) need Read in the selected project, and `ask`, `query`, `dry_run` and `feedback` need Query. A restricted project the user is not a member of, or a project outside an API key's list, is refused before any tool runs; a call without a signed-in request is refused outright. Tool calls over `/beacon/mcp` decide the caller's projects by the same rules (see [Overview](#overview)).

The companion **MCP Learning** page (`/beacon/mcp-learning`) shows the learning loop at work: usage signals, success rate, and learned schema patterns awaiting approval. Any stored documentation patch can also be applied or rejected there.

## Calling the endpoint directly

A client without native Streamable HTTP support can POST JSON-RPC messages to `/beacon/mcp` with the `Authorization: Bearer` header on every request. The request shapes, headers and session handling are in [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/).

## Troubleshooting

**"No projects are available to you" error**: the caller reaches no project: none of the API key's projects is one its owner may still use, a mapped Entra user is not a member of the restricted projects it is configured for, or the call came from a browser session (use the [MCP Playground](#mcp-playground)). Check the key's projects and the owner's or user's project memberships.

**"Access denied: you do not have access to project N"**. The project named in `project_id` is not one the caller reaches (see [Overview](#overview)).

**"Access denied: you may read project N but not run ad-hoc queries in it"**. The caller is a Viewer in the project: it reads the catalog, but `ask`, `query` and `dry_run` need an Editor role there (and, in a restricted project, an Editor membership). Saved-query tools stay available to Viewers.

**"Multiple projects available (IDs: …)"**. The caller reaches several projects. Pass `project_id` on every call, or restrict the API key to a single project.

**SQL refused by the read-only gate**. The statement is not a single read-only query (an `INSERT`, `UPDATE`, DDL, a second statement), or it names another database than the data source's own (`cross_database`), or it reads the engine's catalog, statistics or query history (`system_catalog`). Run `dry_run` to see every issue with its gate and code.

**Connection drops, or "session not found" behind a load balancer**: streaming connections may be interrupted by proxies, and an initialize-handshake client's session lives on one instance and closes after 30 minutes idle. Re-initialize against `/beacon/mcp` to start a fresh session, and give multi-instance deployments session affinity (see [Connection details](#connection-details)).

**Authentication fails**: Verify your API key starts with `sk-sem_`, has not expired, and has not been revoked. Check the `Authorization: Bearer` header format.

**"Permission denied: this credential does not have the Execute scope"**. The key (or Entra caller) has only `Read`, or it has `Execute` but its owner no longer has the Editor role or above. Use an `Execute` key of a user with write permission.

**Answers miss obvious joins or misread a business term**. The model can only use what it was given. Check the [Knowledge Base guide](https://moberghr.github.io/beacon/mcp/knowledge-base/): register the join path, add the glossary term, or promote a correct answer to a golden example.

## Upgrading from 4.5: project access

MCP now uses the same [project access](https://moberghr.github.io/beacon/security/authorization/#project-access) as the REST API and the playground, decided on every call (see [Overview](#overview) and the full [upgrade notes](https://moberghr.github.io/beacon/security/authorization/#upgrading-from-45-project-access)):

- **API keys without a project list** now reach their owner's projects on MCP; before, they reached no project there. This widens what such keys can do over MCP. Re-issue them with a project list to confine them. Keys with a list reach only the listed projects their owner may still use.
- **Mapped Entra user callers** reach a restricted project only while their Beacon user is a member of it, and nothing while their user is disabled or archived; a user caller without a Beacon user reaches its configured open projects only. System callers keep their configured projects. See [Entra ID callers on MCP](https://moberghr.github.io/beacon/mcp/entra-callers/).
- **Saved-query tools** are listed only in projects where the caller may run saved SQL, and each run is re-checked right before its SQL: when a data source the version reads is no longer linked to the project, or the caller may no longer run saved SQL there, the call fails without running anything. **Host endpoint tools** need the caller to read the host's project.
- **A call without an authenticated caller is refused**, over HTTP or over a transport without an HTTP request (such as a host-wired stdio transport), and such a call reaches no project. Browser sessions are refused on `/beacon/mcp`: the route and the MCP layer take an API key or a mapped bearer token only. A refusal at the route answers `403` before any MCP request is read, so it leaves no audit row; a `tools/call` refused inside the MCP layer is audited.
- **The SQL-carrying tools need Query in the project**, as in the playground: `ask`, `query` and `dry_run` (and `feedback`) answer "Access denied: you may read project N but not run ad-hoc queries in it." to a caller that is only a Viewer there, while the catalog tools still answer it. Mapped or provisioned Entra users keep the default `Viewer` role (`Beacon:Mcp:Callers:Users:DefaultRoleName`) and so read the catalog only: give them the Editor role (and, in a restricted project, an Editor membership) to run SQL tools. API keys are already capped by their owner's role.
- **Error messages** no longer mention API keys: "Access denied: you do not have access to project N." and "No projects are available to you."
- **Audit rows and signals** record the caller's Beacon user id. API keys and mapped callers were already recorded so; a signed-in session (the playground) was recorded by its external id, with no user when that id is not a number, so it could not rate the signals of its own `ask` calls. Now it can.
- **Eval runs without a project** are refused: the run is marked `Failed` before any case runs.
- **Database names must be the data source's own.** SQL naming another database, catalog or project than the one the data source's connection configures is refused (`cross_database`), and a data source without a configured database accepts no database qualifier. SQL the gate cannot parse is refused even with read-only enforcement off.
- **The engine's catalog is not queryable.** SQL reading `information_schema`, `pg_catalog`, `sys` or another catalog, statistics or query-history relation is refused (`system_catalog`); use `get_context` for tables and columns.
- **Shared data sources take the strictest guardrails** of the live projects that link them: a project with a looser row limit, PII, read-only, value-grounding or sample-value setting gets the stricter one on a data source it shares.
- **Sample values and value grounding are off by default** for a new installation, and the deployment lock `Beacon:Mcp:ForbidRowDataToLlm` keeps all row data from the LLM provider; see [What reaches the LLM provider](#what-reaches-the-llm-provider).
- **`get_query_context` runs live value probes only for callers who may query the project**; a Viewer's context resolves values from the stored samples alone.

## See also

- [Knowledge Base & Grounding](https://moberghr.github.io/beacon/mcp/knowledge-base/): what grounds generated SQL, and how to curate it
- [API Keys](https://moberghr.github.io/beacon/security/api-keys/): scopes, project restrictions, and rotation
- [Entra ID callers on MCP](https://moberghr.github.io/beacon/mcp/entra-callers/): user and system JWT callers, their projects, scope and audit
- [Admin Settings](https://moberghr.github.io/beacon/security/admin-settings/): LLM provider configuration
- [AI Integration](https://moberghr.github.io/beacon/features/ai-integration/): documentation generation that feeds the MCP catalog tools


---

# Connect AI clients

> Expose Beacon's MCP server from your host, decide how AI clients authenticate (API keys or Entra ID callers), publish discovery metadata, and connect Claude Code, Claude, Cursor and VS Code.

Source: https://moberghr.github.io/beacon/mcp/connect/

Beacon's MCP server lets AI assistants explore and query your data through eight tools, always project-scoped, read-only, masked and audited. This page wires it into your host and connects clients. The tools themselves are described on [MCP server](https://moberghr.github.io/beacon/mcp/server/).

## Wiring

```csharp
builder.Services.AddBeaconApiAuthorization();   // registers the Execute-scope policy MCP needs
builder.Services.AddBeaconAI(builder.Configuration);
builder.Services.AddBeaconMcp();
// …
app.UseMcpResourceMetadataChallenge();           // before UseBeaconAuthentication
app.UseBeaconAuthentication();                   // API keys, JWT callers (when enabled), the cookie
app.UseAuthorization();
app.MapBeaconMcp();                              // /beacon/mcp, Execute scope required
app.MapMcpDiscovery();                           // anonymous discovery documents
```

| Item | Value |
|---|---|
| Endpoint | `/beacon/mcp` |
| Transport | Streamable HTTP |
| Authentication | `Authorization: Bearer sk-sem_…` (API key) or `Authorization: Bearer <Entra access token>` |
| Required scope | **Execute** (a Read-scoped key gets 403) |
| Built-in tools | All eight by default; `AddBeaconMcp(x => …)` chooses (`BuiltInTools`), turns saved-query tools off (`SavedQueryTools = false`) and replaces the server instructions (`ServerInstructions`) and the server card's description (`ServerDescription`). A tool left out is neither listed nor callable, and the server card lists only the ones served. |
| Sessions | Clients that start with `initialize` (most clients today) get an `Mcp-Session-Id` (idle timeout 30 minutes) and need **session affinity** when you run several instances. Clients speaking the stateless protocol revision `2026-07-28` need no session. A request that is neither (no session and no `2026-07-28` metadata) is refused with 400. |

`MapBeaconMcp()` maps the route with the Execute-scope policy, and two filters inside the MCP layer refuse any caller without Execute even if the route were mapped with a weaker policy. Do not map it yourself with `MapMcp(...)` and a bare `RequireAuthorization()`.

:::note[Browser sessions]
A signed-in browser session, Beacon's or your host's, is refused on `/beacon/mcp` with `403`: the route takes an API key or a mapped bearer token only. People try the tools in the UI instead: **MCP playground** (`/beacon/mcp-playground`).
:::

## Public URL and discovery

Clients that connect by URL (claude.ai, Claude Desktop connectors, VS Code) discover how to authenticate from three anonymous documents:

| Path | Content |
|---|---|
| `/.well-known/oauth-protected-resource` | RFC 9728 protected-resource metadata: `resource` (`{base}/beacon/mcp`), `scopes_supported`, `authorization_servers`, bearer in header only |
| `/.well-known/oauth-protected-resource/beacon/mcp` | The same, at the path-inserted location |
| `/.well-known/mcp/server-card.json` | Server card: name, transport, endpoint, authentication, the eight built-in tools |

Every 401 from `/beacon/mcp` carries `WWW-Authenticate: Bearer resource_metadata="{base}/.well-known/oauth-protected-resource"`.

**Set `Beacon:PublicBaseUrl`** (for example `https://beacon.example.com`) in every proxied or public deployment. Without it, the documents and challenges are built from the request's `Host` header, which the client controls; Beacon warns at startup.

To point OAuth-capable clients at your identity provider:

```json
{
  "Beacon": {
    "PublicBaseUrl": "https://beacon.example.com",
    "Mcp": {
      "ProtectedResource": {
        "AuthorizationServers": [ "https://login.microsoftonline.com/<tenant-id>/v2.0" ],
        "ScopesSupported": [ "api://<beacon-app-id>/mcp.access" ]
      }
    }
  }
}
```

| Key | Default | What it does |
|---|---|---|
| `AuthorizationServers` | None | Issuers published as `authorization_servers` (each must equal the server metadata's `issuer`). Takes precedence. |
| `DeriveAuthorizationServerFromJwtIssuer` | `false` | Publish the bearer-JWT issuer instead (prefers an Entra `/v2.0` issuer). |
| `ScopesSupported` | `["Execute"]` | Scopes published to clients. Set your identity provider's scope when you publish an external authorization server, and add `offline_access` so clients get a refresh token instead of asking people to sign in every hour (Beacon warns at startup when an `api://` scope is published without it). A configured list replaces the default whole. |
| `ResourceName` | `Beacon MCP` | The `resource_name` an MCP client shows for the server, also the server card's `name` and the server name sent at `initialize` (`Beacon` when empty). A host serving Beacon as its own MCP server names its application. |
| `ResourceDocumentation` | Beacon's MCP docs | The `resource_documentation` URL, also the server card's `documentation`. Anything but an absolute https URL logs a startup warning. |

Code can complete these settings too: `services.PostConfigure<McpProtectedResourceOptions>(...)` runs after the configuration is bound, on every request.

With none of these set and OIDC enabled, Beacon publishes its OIDC authority.

## Option A: API keys

The simplest path: each person creates a key for themselves.

1. In **API keys** (`/beacon/api-keys`), create a key with the **Execute** scope, limited to the projects the assistant should use, with a short expiry. (Scripts can do the same through `POST /beacon/api/api-keys`; see [headless first run](https://moberghr.github.io/beacon/integrate/authentication/#headless-first-run-scripts-and-ci).)
2. Copy it. It is shown once.
3. Configure the client with `Authorization: Bearer sk-sem_…` (below).

The key acts as its owner, only in the key's projects, capped by the owner's current role: reading tools and saved-query tools work for any owner, while `query` and `ask` need an owner who can query (Editor or above). When a key reaches several projects, the assistant passes `project_id` on each call.

## Option B: Entra ID callers

For organisation-wide access without personal keys, Beacon accepts Entra access tokens on `/beacon/mcp` and maps each token to an **MCP caller** with projects and a scope. Two kinds:

- **Users**: delegated (on-behalf-of) tokens from a client app your users sign in to; optionally auto-provisioned as Beacon users.
- **Systems**: app-only tokens of a known application or service principal.

```csharp
builder.Services.AddBeaconJwtAuthentication(jwt =>
{
    jwt.EnableBearerAuthentication = true;
    jwt.Validation.JwksEndpoint = "https://login.microsoftonline.com/<tenant-id>/discovery/v2.0/keys";
    jwt.Validation.ValidIssuers = [ "https://login.microsoftonline.com/<tenant-id>/v2.0" ];
    jwt.Validation.ValidAudiences = [ "api://<beacon-app-id>" ];
});
```

```json
{
  "Beacon": {
    "Mcp": {
      "Callers": {
        "AllowedTenants": [ "<tenant-id>" ],
        "Users": {
          "Enabled": true,
          "RequiredGroups": [ "<group-object-id>" ],
          "ProjectIds": [ 4 ],
          "GroupProjects": { "<finance-group-id>": [ 7 ] },
          "DefaultRoleName": "Viewer"
        },
        "Systems": [
          { "Name": "reporting-bot", "ClientId": "<app-client-id>", "ProjectIds": [ 4 ], "Scope": "Execute" }
        ]
      }
    }
  }
}
```

| Key | Default | What it does |
|---|---|---|
| `AllowedTenants` | Required | Tenants whose tokens may map to a caller. Required once any caller is configured. |
| `Users:Enabled` | `false` | Accept delegated user tokens. |
| `Users:RequiredRoles` / `RequiredGroups` | None | The user needs one of them; with neither set, `AllowAnyTenantUser` must be `true`. |
| `Users:ProjectIds`, `Users:GroupProjects` | None | Projects every accepted user gets, plus extra projects per group or app role. |
| `Users:Scope` | `Execute` | `Read` or `Execute`. |
| `Users:AutoProvision` / `DefaultRoleName` | `true` / `Viewer` | Create a Beacon user on first call, with this role (never Admin or a functional role). |
| `Systems[]` | None | `Name`, exactly one of `ClientId` (app-only token) or `ObjectId` (service principal), `ProjectIds`, `Scope`, `HostClaims`. |

An absent `Callers` section admits no JWT caller on MCP; it fails closed. Startup validates the section, including that bearer JWT validation is on with explicit issuers and audiences. Audit rows identify Entra callers by a keyed hash (`Beacon:Mcp:CallerHashKey`, else derived from the encryption key). Full reference: [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/).

## Connect a client

### Claude Code

```bash
claude mcp add --transport http beacon https://beacon.example.com/beacon/mcp \
  --header "Authorization: Bearer $BEACON_MCP_API_KEY"
```

Or share it with your team in the repository's `.mcp.json` (Claude Code expands environment variables):

```json
{
  "mcpServers": {
    "beacon": {
      "type": "http",
      "url": "https://beacon.example.com/beacon/mcp",
      "headers": { "Authorization": "Bearer ${BEACON_MCP_API_KEY}" }
    }
  }
}
```

### Cursor

`~/.cursor/mcp.json` or `.cursor/mcp.json` in the project:

```json
{
  "mcpServers": {
    "beacon": {
      "url": "https://beacon.example.com/beacon/mcp",
      "headers": { "Authorization": "Bearer sk-sem_…" }
    }
  }
}
```

### VS Code

`.vscode/mcp.json`:

```json
{
  "inputs": [
    { "type": "promptString", "id": "beacon-key", "description": "Beacon API key", "password": true }
  ],
  "servers": {
    "beacon": {
      "type": "http",
      "url": "https://beacon.example.com/beacon/mcp",
      "headers": { "Authorization": "Bearer ${input:beacon-key}" }
    }
  }
}
```

### Claude (claude.ai and Claude Desktop)

Add Beacon as a **custom connector** with the URL `https://beacon.example.com/beacon/mcp`. Connectors authenticate with OAuth, so this path needs Option B: publish your Entra authorization server and scope through `Beacon:Mcp:ProtectedResource`, register the client's redirect URI in your Entra app, and map users with `Beacon:Mcp:Callers:Users`. The step-by-step Entra setup is on [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/).

Client configuration formats change; if a snippet does not work, check the client's own documentation. The endpoint, header and transport above are what Beacon needs.

## Check it works

```bash
BEACON=https://beacon.example.com
curl -s $BEACON/.well-known/mcp/server-card.json | jq '.tools[].name'

H=(-H "Authorization: Bearer $BEACON_MCP_API_KEY" -H "Content-Type: application/json" -H "Accept: application/json, text/event-stream")

# 1. initialize: the response header Mcp-Session-Id names the session
SESSION=$(curl -s -D - -o /dev/null "${H[@]}" $BEACON/beacon/mcp \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"1"}}}' \
  | awk -F': ' 'tolower($1)=="mcp-session-id"{print $2}' | tr -d '\r')

# 2. confirm initialisation
curl -s "${H[@]}" -H "Mcp-Session-Id: $SESSION" -H "MCP-Protocol-Version: 2025-06-18" $BEACON/beacon/mcp \
  -d '{"jsonrpc":"2.0","method":"notifications/initialized"}'

# 3. list the tools
curl -s "${H[@]}" -H "Mcp-Session-Id: $SESSION" -H "MCP-Protocol-Version: 2025-06-18" $BEACON/beacon/mcp \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}'
```

A client on the stateless revision skips the session: it sends `MCP-Protocol-Version: 2026-07-28` and `Mcp-Method: tools/list` headers (plus `Mcp-Name: <tool>` on `tools/call`) and puts `{"io.modelcontextprotocol/protocolVersion":"2026-07-28","io.modelcontextprotocol/clientCapabilities":{}}` in `params._meta`.

Then ask the assistant something like *"Use Beacon's get_context to describe my project."*

| Symptom | Cause |
|---|---|
| 400 "A new session can only be created by an initialize request" | The client sent a request without a session: start with `initialize` (or use the stateless `2026-07-28` revision). |
| 401 with `resource_metadata` | No or invalid credential. |
| 403 at the door | The key has the Read scope, the JWT caller is not mapped, or a browser session is calling. |
| "Permission denied: this credential does not have the Execute scope" | Same, reported as a tool error. |
| "No projects are available to you." | The key or caller reaches no project. |
| "Access denied: you may read project N but not run ad-hoc queries in it." | Viewer-level access: `query`/`ask` need Query (Editor). |

## Add your own tools

- Publish an approved saved query as an MCP tool `q_<name>` without code: [Saved query tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/).
- Expose endpoints of your own API as `api_<name>` tools: [Extending Beacon](https://moberghr.github.io/beacon/integrate/extending/#your-endpoints-as-mcp-tools).

## Governing MCP

Per-project MCP settings (row limits, read-only, PII detection, content retention, learning) live on **MCP settings** (`/beacon/mcp-settings`), editable by a SecurityAdmin and readable by an Auditor. Deployment locks (`Beacon:Mcp:ForceReadOnly`, `ForcePiiDetection`, `ForceNoContentRetention`, `ForbidRowDataToLlm`) and ceilings (`Beacon:Mcp:Ceilings`) bind them from configuration. Every call is audited; see [MCP observability & audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/).


---

# Entra ID callers on MCP

> Let an agent platform call Beacon's MCP endpoint with Microsoft Entra ID tokens, either on behalf of a user or as a system identity, with projects and scope decided by Beacon configuration.

Source: https://moberghr.github.io/beacon/mcp/entra-callers/

Besides [API keys](https://moberghr.github.io/beacon/security/api-keys/), the [MCP server](https://moberghr.github.io/beacon/mcp/server/) at `/beacon/mcp` accepts **Microsoft Entra ID bearer tokens**. This is how an agent platform calls a Beacon that is embedded in a host application. It supports two modes, and both are first-class:

| Mode | Token | Identified by |
|------|-------|---------------|
| **User** | A delegated token issued on behalf of the user (it carries `scp`) | The user's `oid`, plus `roles` and `groups` |
| **System** | An app-only token from the client-credentials flow (`idtyp=app`, or no `scp` + `roles` + `oid == sub`) | The calling app's `azp` / `appid` |
| **System** (service user) | A token for a dedicated Entra service user | The service user's `oid` |

The token only proves **who** is calling. **What** the caller may do (which projects, and which scope) comes from Beacon configuration. A token cannot grant itself projects or a scope: any `allowed_projects`, `scope`, `auth_method`, `api_key_id` or `caller_*` claim inside the token is discarded.

## How a call is resolved

1. The bearer token is validated (signature against the JWKS endpoint, issuer, audience and lifetime). The JWKS is
   cached: it is fetched once and refreshed hourly, or earlier (at most once every 5 minutes) when a token is
   signed with a key id the cached set does not hold (key rotation). A failed refresh keeps the last good keys for at
   most 24 hours after the last successful fetch; after that, bearer requests answer `503` until a fetch succeeds.
2. On `/beacon/mcp`, Beacon passes the validated claims to `IMcpCallerMapper`. The default `ConfiguredMcpCallerMapper`
   first rejects the token unless:
   - its `tid` is one of `AllowedTenants` (a token without `tid` is rejected);
   - its `aud` is one of the bearer audiences Beacon validates (checked again here, as defense in depth);
   - it is an **access token** of a known kind (see [Token classification](#token-classification)); ID tokens and
     tokens that cannot be classified are rejected.

   It then checks, in this order:
   1. A configured **system** whose `ObjectId` equals the token's `oid` (a service user or a service principal).
   2. A configured **system** whose `ClientId` equals `azp`/`appid`, but **only for app-only tokens**. A delegated user token (or an ID token) issued to the same app also carries its `azp`, so it never matches a `ClientId` system.
   3. **User mode**, for delegated tokens, when it is enabled and the caller has one of the required roles or groups
      (or `AllowAnyTenantUser` is set).
3. A matched caller gets the Beacon claims that API keys already use: `allowed_projects`, `scope` (`Read` or `Execute`), and the Beacon user id as the name identifier when the user is provisioned. It also gets `caller_kind` and `caller_hash`.
4. The route's Execute-scope policy then applies to JWT callers exactly as it does to API keys. **`Read` callers and unknown callers get `403`**, and `Execute` callers pass.

A caller that is neither a configured system nor an accepted user is **unknown**. It gets no projects and no scope, and it is rejected. This fails closed.

The mapped principal carries `auth_method=mcp_caller`, so the Execute-scope rules for scoped callers apply to it as they do to API keys. A browser session (cookie or SSO, Beacon's or the host's) is not a scoped caller and is refused on `/beacon/mcp` with `403`; browsers use the [MCP Playground](https://moberghr.github.io/beacon/mcp/server/#mcp-playground).

### What a caller reaches

The configured projects are what a caller is mapped to. What it reaches in them is decided on every call by Beacon's [project access](https://moberghr.github.io/beacon/security/authorization/#project-access), as for API keys and signed-in users:

| Caller | Reaches |
|--------|---------|
| System | its configured projects, restricted or not |
| User with a Beacon user (`AutoProvision`) | its configured projects within its Beacon user's access: open projects, and a restricted project only while the user is a member of it, at the lower of its membership role and its Beacon roles |
| User with a disabled or archived Beacon user | nothing |
| User without a Beacon user (user management off, or `AutoProvision` off) | its configured **open** projects only |

A membership granted or revoked, a project restricted, or a user disabled takes effect on the caller's next call. Provisioned users get no membership automatically: add them as members of the restricted projects they are configured for (`DefaultRoleName` gives them their global role, `Viewer` by default).

What a user caller with a Beacon user may **do** in a project follows that user's role there, capped by the caller's scope: the catalog tools (`get_context`, `search`, `get_documentation`, `get_query_context`) need Read; `ask`, `query`, `dry_run` and `feedback` work ad hoc and need an Editor role there: the user's global role Editor or above and, in a restricted project, an Editor membership (or project admin). A user provisioned with the default `Viewer` role therefore reads the catalog and runs saved-query tools, but its SQL tools answer "Access denied: you may read project N but not run ad-hoc queries in it." (audited). Raise `DefaultRoleName`, or the user's role, to let it run SQL. System callers, and user callers without a Beacon user, are capped by their scope only (`Execute`: SQL tools allowed).

Provisioning therefore changes what the same person can do. A user caller **without** a Beacon user acts within its configured open projects with the SQL tools (its `Execute` scope allows them); once provisioned, it gets its stored role instead. With the default `Viewer`, this is the catalog only. Both are in the administrator's hands: the caller configuration (its projects and scope) decides the first, `DefaultRoleName` and the user's roles the second. When user callers are configured and `DefaultRoleName` is `Viewer`, Beacon logs this once at startup (Information).

Other routes (`/beacon/api/*`) accept a bearer token only when it names an **existing, enabled, external Beacon
user** by its subject (`sub`) and issuer, or a pre-registered user's subject when only one issuer is configured. A user
that MCP user provisioning created (keyed on `oid` within the tenant) gets **no** REST session from a token. The
session is that user, with the user's **Beacon roles**; no token claim (roles included) reaches it. Any other token gets
`401` with `WWW-Authenticate: Bearer error="invalid_token"`. A configured system identity therefore reaches only
`/beacon/mcp`, where its projects and scope apply.

On `/beacon/mcp` the principal carries only the caller's name, e-mail and display name from the token, plus what the
mapper decides: token roles, groups, `beacon:*` claims and the token's own subject never reach it. Until first-run
setup has created the super admin, `AutoProvision` creates no user and such a caller is rejected.

### Token classification

"No `scp`" alone does not make a token app-only: an ID token, or any other token without `scp`, carries the
requesting app's `azp` too and would otherwise be taken for that app's system identity. The mapper therefore needs a
positive signal:

| Token | Classified as |
|-------|---------------|
| Has `nonce` | ID token → **rejected** |
| `idtyp = app` and no `scp` | App-only |
| `idtyp = app` and `scp` | Contradictory → **rejected** |
| Has `scp` | Delegated (user) |
| No `scp`, has `roles`, and `oid == sub` (a service principal's own token) | App-only |
| Anything else (e.g. neither `scp` nor `roles`) | **Rejected** |

**Entra app registration requirement.** For every system that calls with the client-credentials flow, either add the
optional access-token claim **`idtyp`** to Beacon's app registration (Token configuration → Add optional claim →
Access → `idtyp`; recommended), or assign the calling app at least one **app role** on Beacon's app registration so
its tokens carry `roles`. A client-credentials token with neither is rejected.

## Configuration

Bearer JWT validation has to be on. In the sample host, turn it on by setting `Beacon:Authentication:Oidc:Enabled` and `Beacon:Authentication:Oidc:McpJwksEndpoint`; the sample then validates the issuer against the OIDC `Authority` and the audience against the OIDC `ClientId`, which may not match the v1/v2 issuer and audience pairs Entra puts in its tokens. A host that configures `Callers` usually sets `ValidIssuers` and `ValidAudiences` itself, as below. In your own host, call `AddBeaconJwtAuthentication(x => { x.EnableBearerAuthentication = true; ... })` and `app.UseBeaconJwtBearerAuthentication()`. The token audience must be Beacon's app registration.

**Issuer and audience are mandatory once any caller is configured.** Entra's JWKS signs tokens for every tenant and
every resource, so a valid signature proves nothing about who the token is for. When `Users:Enabled` is on or any
`Systems` entry exists, the host refuses to start unless bearer JWT authentication is enabled, `Validation.ValidIssuer`
(or `ValidIssuers`) and `Validation.ValidAudience` (or `ValidAudiences`) are set, and `ValidateIssuer` /
`ValidateAudience` are left on:

```csharp
builder.Services.AddBeaconJwtAuthentication(x =>
{
    x.EnableBearerAuthentication = true;
    x.Validation.JwksEndpoint = "https://login.microsoftonline.com/<tenant-id>/discovery/v2.0/keys";
    x.Validation.ValidIssuers =
    [
        "https://login.microsoftonline.com/<tenant-id>/v2.0",   // v2 access tokens
        "https://sts.windows.net/<tenant-id>/"                  // v1 access tokens
    ];
    x.Validation.ValidAudiences = ["api://<beacon-app-client-id>", "<beacon-app-client-id>"];
});
```

The callers are configured under `Beacon:Mcp:Callers`:

```jsonc
{
  "Beacon": {
    "Mcp": {
      "Callers": {
        "AllowedTenants": ["<entra-tenant-id>"],          // required once any caller is configured
        "Users": {
          "Enabled": true,
          "RequiredRoles": ["Beacon.Mcp.User"],          // any-of, combined with RequiredGroups
          "RequiredGroups": ["<entra-group-object-id>"],  // empty + empty fails startup...
          "AllowAnyTenantUser": false,                   // ...unless this is set to true explicitly
          "ProjectIds": [1],                             // granted to every accepted user
          "GroupProjects": {                             // group id or app role -> more projects (additive)
            "<entra-group-object-id>": [2, 3],
            "Beacon.Finance": [4]
          },
          "Scope": "Execute",                            // Read | Execute
          "AutoProvision": true,                         // find or create a Beacon user keyed on oid
          "DefaultRoleName": "Viewer"                    // global role for a newly provisioned user
        },
        "Systems": [
          {
            "Name": "agent-platform",
            "ClientId": "<agent-platform-app-client-id>", // app-only token (azp / appid)
            "ProjectIds": [1, 2],
            "Scope": "Execute",
            "HostClaims": [ { "Type": "permission", "Value": "ViewReports" } ]
          },
          {
            "Name": "routine-runner",
            "ObjectId": "<service-user-object-id>",      // dedicated Entra service user (oid)
            "ProjectIds": [2],
            "Scope": "Execute"
          }
        ]
      }
    }
  }
}
```

| Setting | Default | Meaning |
|---------|---------|---------|
| `AllowedTenants` | empty | Entra tenant ids (`tid`) whose tokens may be mapped. **Required and non-empty** once `Users:Enabled` is on or any system is configured |
| `Users:Enabled` | `false` | Accept delegated user tokens at all |
| `Users:RequiredRoles` / `RequiredGroups` | empty | Any-of across both lists. Both empty fails startup unless `AllowAnyTenantUser` is `true` |
| `Users:AllowAnyTenantUser` | `false` | Explicit opt-in to admit **every** user of the allowed tenants when no role or group is required |
| `Users:ProjectIds` | empty | Projects every accepted user is configured for; a restricted one is reached only by its members (see [What a caller reaches](#what-a-caller-reaches)) |
| `Users:GroupProjects` | empty | Additional configured projects per group object id or app role, added together |
| `Users:Scope` | `Execute` | `Read` callers cannot reach `/beacon/mcp` |
| `Users:AutoProvision` | `true` | Create or find a Beacon user (`ExternalId` = `oid`, provider `entra:{tid}`) so audit rows carry a user and project memberships apply. A disabled or archived Beacon user is rejected; within the one minute a caller's user is cached, its calls reach no project instead. This needs user management to be enabled; without it the caller is accepted with no Beacon user |
| `Systems[]:ClientId` / `ObjectId` | none | **Set exactly one per entry.** The host refuses to start otherwise |
| `Systems[]:HostClaims` | empty | The permission set this identity gets inside the host application. It is carried on the caller for in-process API dispatch |

Validation runs at startup. A system entry with both `ClientId` and `ObjectId`, or with neither, a duplicate name, or a non-positive project id, stops the host with a message that names the entry. So do, once any caller is configured: an empty `AllowedTenants`, bearer validation without issuer or audience, and user mode with no role or group requirement and no `AllowAnyTenantUser`. With user mode and `AutoProvision` on, `DefaultRoleName` must be set and must not be the Admin role or the role of a functional permission (every provisioned user would administer Beacon).

With `AutoProvision`, two first requests from the same new user can race to create the Beacon user; the loser
re-reads the row the winner created instead of failing.

:::caution[Group overage]
Entra leaves out the `groups` claim when a user belongs to more than about 200 groups (the "overage" case). For large tenants, prefer **app roles** in `RequiredRoles` and `GroupProjects`, or assign the groups to the app registration so the token only lists relevant groups.
:::

## Connect Claude (claude.ai / Claude Desktop custom connector)

Claude's custom connectors connect by URL and sign the user in with OAuth. Beacon does not run an OAuth server here:
Claude gets a delegated **Entra** token for the user and calls `/beacon/mcp` with it, and Beacon validates the token
and maps the user like any other [user-mode caller](#how-a-call-is-resolved). Claude learns where to sign in from
Beacon's [discovery documents](https://moberghr.github.io/beacon/mcp/server/#discovery-endpoints):

1. `POST /beacon/mcp` without a token answers `401` with `WWW-Authenticate: Bearer resource_metadata="https://<host>/.well-known/oauth-protected-resource"`.
2. That document names the authorization server (`authorization_servers[0]`, the Entra v2 issuer), the scopes to request
   (`scopes_supported`) and the resource (`resource`, the canonical `https://<host>/beacon/mcp` URL).
3. Claude reads Entra's metadata at `https://login.microsoftonline.com/<tenant-id>/v2.0/.well-known/openid-configuration`,
   signs the user in, and requests a token for those scopes, sending the resource URL as the RFC 8707 `resource`
   parameter.

By default the document lists Beacon's internal scope name (`Execute`) and no authorization server unless
Beacon's own OIDC login is on. Neither is something Entra understands. Configure it under
`Beacon:Mcp:ProtectedResource`.

### 1. The API app registration

This is the app registration whose tokens Beacon validates (`Validation.ValidAudiences`). In an embedded setup, usually
the host's API registration.

- **Manifest:** set `api.requestedAccessTokenVersion` to `2` (`accessTokenAcceptedVersion: 2` in the older AAD Graph
  manifest format). Do this first: Entra only lets a v2 app register an HTTPS Application ID URI.
- **Expose an API → Application ID URI:** keep `api://<api-client-id>` and **add the exact MCP URL**,
  `https://<host>/beacon/mcp`, to `identifierUris`. MCP clients send that URL as the `resource` parameter, and Entra
  does *not* ignore it: it must match one of the app's Application ID URIs character for character (no trailing
  slash), or the token request fails with `AADSTS9010010`. Entra may require the host name to be a verified domain of
  the tenant before it accepts an HTTPS Application ID URI.
- **Expose an API → Add a scope:** `mcp.access` (delegated; admins and users, or admins only). Clients request it fully
  qualified, `api://<api-client-id>/mcp.access`. The scope and the `resource` must belong to the same app registration.
- **App roles** (recommended): e.g. `Beacon.Mcp.User`, assigned to the users or groups who may use the connector, and
  listed in `Users:RequiredRoles`.

The token Claude receives is a v2 access token: `iss` is `https://login.microsoftonline.com/<tenant-id>/v2.0`, `aud`
is the API's **client id GUID** (not the `api://` URI and not the MCP URL), and `scp` is `mcp.access`. Keep both audience
forms in `ValidAudiences` so v1 callers of the same API keep working.

### 2. A client app registration for Claude

Entra has no Dynamic Client Registration, so Claude cannot register itself. Create a separate registration for it:

- **Authentication → Add a platform → Web**, redirect URI `https://claude.ai/api/mcp/auth_callback`.
- **Certificates & secrets:** create a client secret (it is a confidential client). Note its expiry. The connector
  stops working when it expires.
- **API permissions → Add a permission → APIs my organization uses →** the API registration → **Delegated** →
  `mcp.access`, then **Grant admin consent** for the tenant.

### 3. Beacon configuration

```csharp
builder.Services.AddBeaconJwtAuthentication(x =>
{
    x.EnableBearerAuthentication = true;
    x.Validation.JwksEndpoint = "https://login.microsoftonline.com/<tenant-id>/discovery/v2.0/keys";
    x.Validation.ValidIssuers = ["https://login.microsoftonline.com/<tenant-id>/v2.0"];
    x.Validation.ValidAudiences = ["<api-client-id>", "api://<api-client-id>"];
});
```

```jsonc
{
  "Beacon": {
    "PublicBaseUrl": "https://<host>",                  // resource = https://<host>/beacon/mcp
    "Mcp": {
      "ProtectedResource": {
        "AuthorizationServers": ["https://login.microsoftonline.com/<tenant-id>/v2.0"],
        // or, instead of listing it: "DeriveAuthorizationServerFromJwtIssuer": true
        "ScopesSupported": ["api://<api-client-id>/mcp.access"]
      },
      "Callers": {
        "AllowedTenants": ["<tenant-id>"],
        "Users": { "Enabled": true, "RequiredRoles": ["Beacon.Mcp.User"], "ProjectIds": [1] }
      }
    }
  }
}
```

| Setting (`Beacon:Mcp:ProtectedResource`) | Default | Meaning |
|---------|---------|---------|
| `AuthorizationServers` | empty | Issuers published as `authorization_servers`, verbatim and in order (clients use the first). Use the **v2** issuer: it equals the `issuer` in Entra's v2 metadata and the `iss` of the tokens. Wins over everything below |
| `DeriveAuthorizationServerFromJwtIssuer` | `false` | With no explicit list, publish the bearer JWT issuer (`Validation.ValidIssuer(s)`). With several, the one ending in `/v2.0` is chosen; templated issuers (`{tenantid}`) are skipped |
| `ScopesSupported` | empty | Scopes published in `scopes_supported` (and the server card) instead of `Execute` |
| `ResourceName` | empty | The `resource_name` clients show (and the server card's `name`, the server name at `initialize`); empty publishes `Beacon MCP` |
| `ResourceDocumentation` | empty | The `resource_documentation` URL (and the server card's `documentation`), https; empty publishes Beacon's MCP docs |

With none of these set the document lists Beacon's OIDC authority when SSO is on, and the `Execute` scope. An explicit authorization server without `ScopesSupported`, or derivation with no usable
issuer, is logged as a warning at startup. The `WWW-Authenticate` challenge, the `resource` in the metadata and the
server card's `endpoint` and `resource_metadata` all derive from the same `PublicBaseUrl` (or the forwarded request
host when it is unset), so they always agree.

### 4. Add the connector in Claude

:::caution[Verify against Anthropic's current connector docs]
This repository cannot test Claude's UI, so the steps below are a best reading of it at the time of writing. Check
Anthropic's current custom-connector documentation for menu names, the redirect URI, and plan or admin requirements.
:::

- In claude.ai, **Settings → Connectors → Add custom connector** (an organization owner may have to add it for the
  organization first). Enter the server URL, `https://<host>/beacon/mcp`.
- Open **Advanced settings** and enter the **OAuth Client ID** and **OAuth Client Secret** of the Claude client
  registration from step 2.
- Connect. Claude opens the Entra sign-in, the user consents (or admin consent already covers it), and the tools appear.
  Claude Desktop uses the connectors of the signed-in claude.ai account.

Things to check when the connection fails, none of which Beacon can see:

- **PKCE advertisement.** Entra's metadata does not publish `code_challenge_methods_supported`, although Entra supports
  PKCE (`S256`). The MCP authorization spec tells clients to refuse an authorization server that does not advertise
  it; whether Claude enforces that for Entra is for Anthropic's docs to say.
- **Refresh tokens.** If Claude asks the user to sign in again every hour, add `offline_access` to `ScopesSupported`
  (clients request the listed scopes verbatim).
- **`AADSTS9010010`**: the `resource` URL Claude sent is not an Application ID URI of the API registration, or a
  requested scope belongs to another registration. Compare the `resource` in
  `/.well-known/oauth-protected-resource` with the registration's `identifierUris`.
- **`401` with `error="invalid_token"`**: Beacon rejected the token: check `iss`/`aud` against `ValidIssuers` /
  `ValidAudiences` (a v2 token's `aud` is the client id GUID). A malformed bearer value gets the same `401`, never a `500`.
- **`403`**: the token is valid but the user is not an accepted caller (`Users:Enabled`, `RequiredRoles` /
  `RequiredGroups`, `AllowedTenants`).
- **`503` with `Retry-After`**: Beacon could not fetch the Entra JWKS and has no usable keys (none cached, or the last
  good set is more than 24 hours old). After a failed fetch, the next one is tried 30 seconds later.

## Audit and privacy

Every MCP tool call writes an audit row, as before. For a JWT caller, the row also records:

- `CallerKind`: `User` or `System`.
- `CallerHash`: the lowercase hex HMAC-SHA256 of `{tid}:{subject}`, where the subject is the token's `oid`, else its `azp` / `appid`, else its `sub`. It is stable per identity and never the raw value. The HMAC key is derived for this purpose only from `Beacon:Mcp:CallerHashKey` when it is set, else from `Beacon:EncryptionKey`, so someone who holds the tenant's list of `oid`s cannot reverse the hashes. Without `CallerHashKey`, changing the encryption key changes every caller's hash, so rows written before and after the change no longer group together: before changing the encryption key, set `Beacon:Mcp:CallerHashKey` (a secret) to the key in use now, and the hashes stay the same (see [Configuration → Replacing the Key](https://moberghr.github.io/beacon/getting-started/configuration/#replacing-the-key)).
- `UserId`: the provisioned Beacon user, in user mode.

Logs carry only the caller hash, never the raw `oid`, email or token.

## Replacing the mapper

A host can replace the default mapping. For example, it can map an Entra user to its own admin user and permission set. To do this, register an `IMcpCallerMapper` before `AddBeaconServices`, which registers the default with `TryAdd`, or replace the registration afterwards:

```csharp
builder.Services.AddScoped<IMcpCallerMapper, AcmeMcpCallerMapper>();
```

The mapper receives the raw token claims (`oid`, `tid`, `groups`, and so on) and returns an `McpCaller`, or `null` to reject the caller. `ConfiguredMcpCallerMapper` is public, so a custom mapper can delegate to it and adjust the result. Inject `McpCallerSubjectHasher`, which `AddBeaconServices` registers as a singleton, to produce the same audit hash.

## Upgrading from 4.5: project access

- **User callers need membership on restricted projects.** A user caller reaches a configured project that is restricted only while its Beacon user is a member of it; open configured projects are reached as before. Add the provisioned users as members of the restricted projects they are configured for.
- **User callers without a Beacon user reach only their configured open projects** (before: every configured project).
- **A disabled or archived user's caller reaches nothing from its next call**, also within the minute its mapping is cached.
- **System callers are unchanged**: they reach their configured projects, restricted or not.
- **User callers with a Beacon user need Editor to run SQL tools.** `ask`, `query`, `dry_run` and `feedback` need the user's role in the project to be Editor or above (global role, and membership on a restricted project); the catalog tools need Read only. Users provisioned with the default `DefaultRoleName` `Viewer` now read the catalog but are refused the SQL tools: give them the Editor role (and Editor membership on restricted projects) if they should run SQL.
- **Host endpoint and saved-query tools** follow the same access: host endpoint tools need the caller to reach the host's project, and saved-query tools are listed only where the caller may run saved SQL.
- **New projects are restricted.** A project created after the upgrade by an active Beacon user is restricted, so a user caller configured for it reaches it only once its provisioned Beacon user is a member, with an Editor membership to run SQL tools. Checklist for provisioned users: the default `Viewer` role reads the catalog only; restricted configured projects need membership; the SQL tools need Editor (global role and membership).

## Upgrading from 4.3

The gateway release changes how JWT callers reach MCP. Before upgrading a host that accepts bearer tokens:

- **JWT callers on `/beacon/mcp` now need `Beacon:Mcp:Callers`.** A validated token alone no longer reaches MCP:
  without a matching system or accepted user the caller gets `403`. Configure `Systems` / `Users` for every agent
  platform that calls MCP with a token. API keys are unaffected.
- **Reserved claims are stripped from tokens on every route.** `allowed_projects`, `scope`, `auth_method`,
  `api_key_id` and `caller_*` inside a token are discarded on `/beacon/api/*` too, so a token can no longer carry its
  own project restriction or scope.
- **New mandatory settings once any caller is configured:** `Beacon:Mcp:Callers:AllowedTenants`, bearer
  `Validation.ValidIssuer(s)` and `Validation.ValidAudience(s)` with validation on, and either `RequiredRoles` /
  `RequiredGroups` or `Users:AllowAnyTenantUser = true` for user mode. The host fails at startup with a message that
  names what is missing.
- **App-only tokens need a positive signal**: the `idtyp` optional claim or an app role (see
  [Token classification](#token-classification)).
- **New migrations**, in both providers: `AddMcpAuditCallerIdentity` (audit caller kind/hash),
  `AddHostManagedDataSources`, `AddProjectImportedDocuments`, `AddQueryMcpTools` and `AddHostManagedProjects`
  (`Project.HostManagedKey`). Apply them before the new version starts.

### Hosts whose cookie authentication redirects

A host whose MVC cookie authentication answers an unauthenticated request with a `302` to its login page breaks MCP clients: an MCP
client cannot follow that, and an agent then sees an HTML login page instead of an auth error. Make the cookie
events return status codes for the MCP path:

```csharp
options.Events.OnRedirectToLogin = ctx =>
{
    if (ctx.Request.Path.StartsWithSegments("/beacon/mcp"))
    {
        ctx.Response.StatusCode = StatusCodes.Status401Unauthorized;
        return Task.CompletedTask;
    }

    ctx.Response.Redirect(ctx.RedirectUri);
    return Task.CompletedTask;
};
options.Events.OnRedirectToAccessDenied = ctx =>
{
    if (ctx.Request.Path.StartsWithSegments("/beacon/mcp"))
    {
        ctx.Response.StatusCode = StatusCodes.Status403Forbidden;
        return Task.CompletedTask;
    }

    ctx.Response.Redirect(ctx.RedirectUri);
    return Task.CompletedTask;
};
```


---

# MCP observability and audit

> Every MCP tool call is auditable, correlated to a W3C trace, and optionally exported through OpenTelemetry, without Beacon choosing your telemetry backend.

Source: https://moberghr.github.io/beacon/mcp/observability-and-audit/

Every tool call through the [MCP server](https://moberghr.github.io/beacon/mcp/server/) (the built-in tools, saved-query tools and
[host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/)) writes one `McpAuditLog` row and emits one structured log
event. That includes calls that fail: a tool's own validation errors (an unresolvable `project_id`, a bad `format`,
an `offset` out of range), project-access refusals, calls refused for a missing `Execute` scope (audited by the scope
filter, before any tool runs) and calls whose project access could not be determined. An unknown `q_<name>` is
audited by the saved-query tools. A tool name that belongs to no tool family is answered as a JSON-RPC error
(`Unknown tool`) and writes no row.

With `Beacon:Mcp:Audit:Required = true`, a result whose row could not be written, or a successful result that
produced no row, is withheld rather than returned; see [Fail-closed audit](#fail-closed-audit). Beacon never picks
an exporter or a backend itself: it tags the MCP SDK's own span and records its own metrics, and a host wires those
into whatever it already uses (OTLP to an ADOT collector, Phoenix, or another backend).

## Configuration

```jsonc
{
  "Beacon": {
    "Mcp": {
      "Audit": {
        "Required": false,               // fail closed: withhold a result whose audit row could not be written
        "RetentionDays": null,           // null or 0 = keep forever (default); 365 recommended; > 0 = purge older rows daily
        "RequestIdHeader": "X-Request-Id", // upstream correlation header; null/empty = read none
        "RequireContent": false          // pin content retention on, so audit content cannot be switched off at runtime
      }
    },
    "Telemetry": {
      "CaptureContent": false            // put the tool input on the current span: double opt-in, see below
    }
  }
}
```

| Setting | Default | Meaning |
|---|---|---|
| `Beacon:Mcp:Audit:Required` | `false` | When `true`, a tool result whose audit row failed to write is replaced with an error instead of being returned. While it is `false` the host logs a Warning at startup: such a call is then recorded only by the `Beacon.Audit` 9100 event (`Persisted=false`) and the audit write failure metric. |
| `Beacon:Mcp:Audit:RetentionDays` | `null` | Days audit rows are kept. `null` or `0` keeps them forever (`365` recommended); a negative value fails options validation at startup. Under `Beacon:Retention:Locked` it must be greater than 0. |
| `Beacon:Mcp:Audit:RequestIdHeader` | `X-Request-Id` | The upstream header read into `UpstreamRequestId`. Must be a valid HTTP header token, or startup fails. Set to `null`/empty to read none. |
| `Beacon:Mcp:Audit:RequireContent` | `false` | When `true`, pins `RetainQueryContent` on globally and for every project, so nobody, an Admin included, can switch the audit's question and SQL text off at runtime. See [Keeping audit content](#keeping-audit-content-the-requirecontent-lock). |
| `Beacon:Telemetry:CaptureContent` | `false` | Puts the tool input on the current span as `beacon.tool.input`, only where the project's own content retention also allows content (double opt-in, never from this setting alone). |

## Knowing the retention jobs run

Beacon schedules no maintenance job itself: the audit purges and the security audit verification run only where the host
schedules them (the sample host does, with Warp recurring jobs `mcp-audit-retention`, `security-audit-retention` and
`security-audit-verify`). Each of those services records a heartbeat (`MaintenanceHeartbeats`, one row per job: its name and
the time it last ran to its end) when it finishes. At startup the host looks the heartbeats up, bounded and best effort, and
logs a Warning for each job that should be running and has no recorded run, or none within 36 hours (6 for the
verification): a purge counts only while its `RetentionDays` is set. A first start has no heartbeat yet and warns once.
`IRetentionScheduleCheck` runs the same check for a host that wants it on a schedule of its own.

## What's on an audit row

`McpAuditLog` (one row per tool call) carries, beyond the tool name, timing, result row count and (redacted, per the
project's retention decision) parameters/error:

| Column | Source |
|---|---|
| `CorrelationId` | Minted by Beacon for every audited call (32 hex characters) and never read from a request: the one identifier on the row, and on the 9100 event, that a caller cannot choose. `null` on rows written before it existed. |
| `TraceId` / `SpanId` | `Activity.Current` (the MCP SDK's `tools/call` span) when one is current; otherwise `null`. |
| `McpSessionId` | The `Mcp-Session-Id` transport header. `null` for stateless clients. |
| `UpstreamRequestId` | Supplied by the caller and not verified: use `CorrelationId` as the identifier Beacon vouches for. The header named by `Beacon:Mcp:Audit:RequestIdHeader`, accepted only when it is at most 128 characters of `[A-Za-z0-9._:-]`; anything else is dropped, which blocks log injection. |
| `ApiKeyId` | The `api_key_id` claim of an API-key caller. No foreign key: the audit must outlive a deleted key. |
| `CallerKind` / `CallerHash` | Set for a mapped [Entra ID caller](https://moberghr.github.io/beacon/mcp/entra-callers/); `null` for API keys and cookie sessions. |
| `ContentHash` | A keyed hash of the call's parameters as received, written **whether or not** the parameters themselves are kept, so calls with the same input can be matched (and a disclosed input checked against the trail) without keeping it. See [Keyed hashes](#keyed-hashes). |
| `DataSourceId`, `ResultRowCount` | For `ask`: the data source and final row count of a single-source answer; `null` for a cross-source answer, whose sources and row counts are on its statements. |

### The statements a call ran

Each audit row also has one `McpAuditStatement` child row per statement the call sent to a data source, in the order
they ran, on the success and the failure path alike, whatever the learning settings. The executors that reach a data
source record them (the query execution service, the cross-source service and the value-grounding probes), and the
pipeline that runs them names why:

| Purpose | Statement |
|---|---|
| `initial` | The first execution of the SQL an `ask` generated. |
| `repair` | An execution of a repaired statement, after an execution error or an empty result. |
| `candidate` | A self-consistency candidate executed to vote on its result. |
| `probe` | A value-grounding probe for a literal of the question. |
| `dry_run` | A validation on the data source without running the statement (`EXPLAIN` and the like), by `dry_run` or before a cross-source step. |
| `source_step` | One data source's statement of a cross-source answer (the in-memory join that combines them reaches no data source and is not a statement). |
| `query` | A statement the `query` tool ran as asked, or one run outside any named purpose. |

A statement row holds its data source, its purpose, `SqlHash` (a keyed hash of the statement text, **always** kept),
`Sql` (the text: **content**, kept only while the call's project retains query content; the content-retention
interceptor strips it otherwise), the tables it names, its row count, its duration and the class of the error it ended
with (never the message; an empty error message counts as no error). Statement rows are deleted with their audit row, so
the retention purge covers them.

A row never reads as the whole of what a call ran when it is not:

- **At most 100 statements a call.** Further ones are counted, not stored: the call's statements then end with a marker
  row of purpose `truncated` (no data source and no text) whose `RowCount` is the number of statements not stored, and a
  Warning names the tool and the count.
- **Table lists.** A statement's table list holds as many names as fit its column; when not all fit, its last entry is
  `[+n more]`. A statement whose tables could not be read from its text has the list `["[unparsed]"]`: unknown, not
  empty. The tables a call's structural parameters name (kept when its content is not) leave these markers out.

### Keyed hashes

`ContentHash` and `SqlHash` are lowercase hex HMAC-SHA256 values under a key derived, for this purpose only (HMAC of a
fixed label), from the secret the MCP caller hashes come from: `Beacon:Mcp:CallerHashKey`, else `Beacon:EncryptionKey`.
An absent input has a hash of its own, apart from the empty text's.
The secret itself is never the key, and nobody without it can test guesses against the stored hashes. The hashes stay
comparable across an encryption-key change while `Beacon:Mcp:CallerHashKey` is set (see
[Replacing the Key](https://moberghr.github.io/beacon/getting-started/configuration/#replacing-the-key)).

### Sealed rows

Every row is sealed as it is saved: an HMAC-SHA256 over its fields and the statements it holds, under the
[security audit trail](https://moberghr.github.io/beacon/security/audit-trail/#the-mcp-audit-rows)'s integrity keys (`Hmac`, `HmacKeyId`, `SealVersion`,
`AuditGuid`). Beacon refuses to change or delete a row or a statement, and the scheduled security audit verification
checks the rows written since its last run, checks again a rolling slice of the rows it checked before, and counts the
rows up to its checkpoint, reporting changed and deleted ones at Error (9111) and writing a checkpoint (9112).
Rows written before seals were introduced are reported as unsealed.

## The `Beacon.Audit` log event catalogue

Every event below is logged at category **`Beacon.Audit`**, structured, with identifiers and counts only, never
parameters, SQL, a question, or a raw error message (only its class, via `McpContentRedactor.ErrorClassOf`). Point
your log sink (CloudWatch, a SIEM) at this category to get the audit trail without reading Beacon's database.

| EventId | Name | Level | When | Notable fields |
|---|---|---|---|---|
| 9100 | `McpToolAudited` | Information | Once per `McpAuditService.LogToolCallAsync` call, including when the row failed to save. | `Tool`, `AuditId` (null if not persisted), `Persisted`, `ProjectId`, `DataSourceId`, `UserId`, `ApiKeyId`, `CallerKind`, `CallerHash`, `DurationMs`, `Rows`, `ErrorClass`, `CorrelationId`, `TraceId`, `McpSessionId`, `UpstreamRequestId` |
| 9101 | `McpToolResultWithheld` | Warning | `Beacon:Mcp:Audit:Required = true` and the audit write for that call failed, or a successful result was not audited (MCP transport and Playground/REST tool runs). | `Tool`, `TraceId` |
| 9102 | `McpAuditPurged` | Information | The retention purge runs with `RetentionDays` set. | Deleted count, cutoff |
| 9103 | `McpAuditRead` | Information | `GET /beacon/api/mcp/audit` is called. | `RequestedByUserId`, `From`, `To`, `ProjectId`, `HasToolFilter` (bool: the admin-supplied `Tool` filter is free text, so only its presence is logged, never the value), `UserId`, `HasCallerHashFilter` (bool, same reasoning), `Page`, `PageSize`, `ReturnedCount` |

The same category carries the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/#shipping-to-a-siem)'s events 9104 to 9114: a copy of every committed security audit event, reads and verifications of the trail, purges, write and verification failures, the checkpoints of its scheduled verification, and the failures (9111) and checkpoints (9112) of the MCP audit rows' verification.

### Row data sent to the LLM provider

Prompts that carry data read from a data source are logged at category
`Beacon.AI.Services.LlmProviders.LlmRowDataEgress`, event **7123** `RowDataSentToLlm`, identifiers only, never a value
or prompt text, and no table is written:

| Level | When | Message fields |
|---|---|---|
| Information | A request carrying row data is handed to the LLM provider (written before the provider answers, since the data may leave even when it then fails). | `Categories`, `DataSourceIds`, `ProjectId`, `CallerId` |
| Warning | The provider then fails that request. | the same, plus `Failure`: the exception's type and error code, never its message |
| Information | A `get_query_context` response returns sample values or value matches to the MCP client. | `Categories`, `DataSourceIds`, `ProjectId`, `CallerId` |

The categories are `sample-values`, `value-matches`, `column-values` (documentation enum discovery), `result-rows`
(the eval judge), `database-errors` (lesson extraction) and `stored-documentation` (business-domain, data-model and
glossary sections, which can list discovered values). `CallerId` is the caller's Beacon user id; background work names
itself: `learning-aggregation`, `documentation-indexing`, `pattern-replay`, an eval run's starting user or `eval-run`,
documentation generation's requesting user; and a caller that cannot be resolved is `unknown` (the lookup failure is
logged as a Warning by type, and never fails the request). See
[What reaches the LLM provider](https://moberghr.github.io/beacon/mcp/server/#what-reaches-the-llm-provider).

## Span attributes and metrics

The MCP SDK already opens one span per `tools/call` (activity source `Experimental.ModelContextProtocol`, tagged
`gen_ai.tool.name`, `mcp.method.name`, `mcp.session.id`, `error.type`, `jsonrpc.request.id`) and records
`mcp.server.operation.duration`. Beacon does not open a duplicate span; it tags the SDK's own current activity. The
SDK's spans are exported only when the host adds them with `AddBeaconMcpSdkTracing()` (see
[Turning tracing on](#turning-tracing-on)):

| Tag | Meaning |
|---|---|
| `beacon.project.id`, `beacon.data_source.id` | The resolved project/data source. |
| `beacon.caller.kind`, `beacon.caller.hash` | The mapped caller (Entra callers only). |
| `beacon.result.rows` | Rows returned, when the tool reports a count. |
| `beacon.error.class` | The redacted error class (never the raw message). |
| `beacon.audit.persisted` | Whether the audit row was written. |
| `beacon.tool.input` | The tool input, truncated to 4000 characters. **Double opt-in only**. see below. |

On the `Beacon` meter:

| Instrument | Kind | Tags |
|---|---|---|
| `beacon.mcp.tool.calls` | Counter | `gen_ai.tool.name`, `beacon.outcome` (`success`\|`error`), `error.type`, `beacon.caller.kind` |
| `beacon.mcp.tool.rows` | Histogram | `gen_ai.tool.name` |
| `beacon.mcp.audit.write_failures` | Counter | `gen_ai.tool.name` |
| `beacon.error_redaction.fallbacks` | Counter | none: a database error that could not be [screened](https://moberghr.github.io/beacon/mcp/server/#database-errors) in time and was replaced as a whole |

Duration is not duplicated on a Beacon instrument; the SDK's own `mcp.server.operation.duration` already covers it.

Tool names can be caller-chosen (an unknown `q_<anything>` saved-query name, for instance). The 9100/9101 `Tool` log
field carries the tool name only when it matches `^[A-Za-z0-9_.:\-]{1,200}$`; anything else is reported as the
constant `<invalid>`. This blocks log injection; logs are not a cardinality problem, so a charset/length bound is
enough there.

The `gen_ai.tool.name` **metric** tag is drawn from a closed set instead: a charset bound alone doesn't fix
cardinality, because a well-formed but caller-chosen name (`q_1`, `q_2`, ... each a distinct saved-query name) would
still open a new time series per name. A built-in tool (the attribute tools in `McpToolCatalog`, plus the fixed
`search_saved_queries` / `run_saved_query` / `search_api` / `call_api` dispatcher names) keeps its own value; an
unknown name starting with `q_` or `api_` (including a write tool's `api_<name>:write` audit name) collapses to the
fixed bucket `q_*` / `api_*`; anything else is `<other>`.
That is a handful of possible values regardless of how many distinct tool names callers invent.

The audit row itself keeps the tool name as before, unredacted.

> The MCP SDK's own meter (also exported by `AddBeaconInstrumentation`) tags its instruments (including
> `mcp.server.operation.duration`) with `gen_ai.tool.name` set to the **raw** caller-chosen name; Beacon does not
> control that tag. A host worried about cardinality on the SDK's own meter can drop or re-bucket that tag with an
> OpenTelemetry [View](https://opentelemetry.io/docs/specs/otel/metrics/sdk/#view) in its `MeterProviderBuilder`.

### LLM completions and ask stages

Beacon opens its own spans on the `Beacon` activity source for the work behind a tool call. **No tag carries
content**: never the prompt, the system prompt, the completion, SQL or the question, whatever
`Beacon:Telemetry:CaptureContent` says. Tags hold names, counts and flags only.

`beacon.llm.completion` (kind Client): one span per LLM completion, opened around the request queue, so it covers the
wait for a slot, the retries and the provider call.

| Tag | Meaning |
|---|---|
| `gen_ai.provider.name` | The configured provider kind (`openai`, `claude`, `azureopenai`, `bedrock`), read from the same configuration snapshot as the provider that serves the call. |
| `gen_ai.request.model` | The configured model (a configuration value), from that same snapshot. |
| `gen_ai.response.model` | The model the provider reports, only when it reads as a model name (at most 128 letters, digits and `. - _ : / @`); absent otherwise. |
| `beacon.llm.operation` | What the completion is for, a fixed name such as `sql.generation`, `sql.self_consistency`, `sql.repair`, `ask.intent`, `ask.routing`, `ask.knowledge_answer`, `doc.indexing`, `learning.lesson`, `documentation.generate`, `actor.think`. Absent when the caller set none. |
| `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens` | Tokens the provider reports. |
| `gen_ai.usage.cache_read.input_tokens`, `gen_ai.usage.cache_creation.input_tokens` | Prompt-cache reads and writes, where the provider reports them (Claude, also on Bedrock); otherwise 0. |
| `beacon.llm.cache_hint` | Whether the request marked any message as a prompt-cache breakpoint. Only the Claude provider acts on it; Bedrock does not request prompt caching yet, and OpenAI and Azure OpenAI cache on their own. |
| `beacon.llm.truncated` | Whether the answer stopped at the token limit. |
| `beacon.outcome` | `success`, `error`, or `cancelled` when the call was cancelled (a cancellation sets no error status and no error type). |
| `error.type`, `beacon.error.root_type` | On an error only: the exception's type name and the innermost exception's type name (a provider wraps its failures, so the root shows the cause). Never a message. The span's status is Error. |

`beacon.ask.<stage>`: one span per stage of an `ask`, tagged `beacon.ask.stage` and counts only. A completion made in
a stage is its child span. A stage that throws is marked like a completion: `beacon.outcome` `cancelled` on a
cancellation, otherwise `error` with error status, `error.type` and `beacon.error.root_type` (a failing self-consistency
vote is marked so although the `ask` goes on with the single candidate).

| Span | Extra tags |
|---|---|
| `beacon.ask.intent` | `beacon.ask.intent` (`DataQuery` or `Knowledge`) |
| `beacon.ask.routing` | `beacon.ask.sources` (data sources considered) |
| `beacon.ask.generation` | `beacon.ask.tables` (distinct tables the candidate reads), `beacon.ask.context_chars` (characters of the schema context sent), `beacon.ask.tables_shown` and `beacon.ask.tables_omitted` (tables the context describes, and tables `Ceilings:AskSchemaContextChars` left out; these three are set before the call, so a failed generation carries them too) |
| `beacon.ask.self_consistency` | `beacon.ask.candidates` (candidates voted on) |
| `beacon.ask.repair` | `beacon.ask.repair.trigger` (`schema`, `lint`, `dry-run`, `execution`, `empty-result`), `beacon.ask.produced_sql` (whether a statement came back) |
| `beacon.ask.execution` | `beacon.ask.statement.purpose` (`initial`, `repair`, `candidate`), `beacon.result.rows`, `beacon.outcome` |

On the `Beacon` meter:

| Instrument | Kind | Tags |
|---|---|---|
| `beacon.llm.calls` | Counter | `gen_ai.provider.name`, `gen_ai.request.model`, `beacon.llm.operation`, `beacon.outcome` (`success`, `error`, `cancelled`), `error.type` (errors only) |
| `beacon.llm.tokens` | Histogram | `gen_ai.provider.name`, `gen_ai.request.model`, `beacon.llm.operation`, `gen_ai.token.type` (`input`, `output`, `cache_read`, `cache_write`); a type with no tokens records nothing |

A completion made outside a request queue's provider (the admin connection test) has no span.

### Content on spans is a double opt-in

`beacon.tool.input` is added to the current span only when **both**:
1. `Beacon:Telemetry:CaptureContent = true`, and
2. the calling project's effective settings retain query content (`ContentRetentionDecision.RetainQueryContent`),
   which a deployment-level `Beacon:Mcp:ForceNoContentRetention` lock always pins off.

Log events and metrics never carry content, regardless of either setting.

The MCP SDK writes an error result's content, or an exception's message, into its span's status description.
`AddBeaconMcpSdkTracing()` clears every such description when the span ends, before the provider's processors and
exporters see it; the status code (`Error`) and `error.type` stay. Keep the SDK's `ModelContextProtocol` log category
above `Trace` in production: at `Trace` the SDK logs whole requests and responses.

## Turning tracing on

Beacon.Api exposes `BeaconTelemetryBuilderExtensions.AddBeaconInstrumentation()` for both `TracerProviderBuilder` and
`MeterProviderBuilder`. It only depends on `OpenTelemetry.Api`; it adds the `"Beacon"` source to the tracer provider
and the `"Beacon"` meter plus the MCP SDK's own `Experimental.ModelContextProtocol` meter to the meter provider.
`AddBeaconMcpSdkTracing()` adds the SDK's `tools/call` spans (the spans Beacon tags) with their status descriptions
cleared; without it those spans, and Beacon's tags on them, are not exported. **Beacon never picks an exporter.** The sample host (`Beacon.SampleProject/Program.cs`) enables OpenTelemetry only when
`OTEL_EXPORTER_OTLP_ENDPOINT` is set:

```csharp
if (!string.IsNullOrWhiteSpace(builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]))
{
    builder.Services.AddOpenTelemetry()
        .ConfigureResource(x => x.AddService(serviceName: builder.Configuration["OTEL_SERVICE_NAME"] ?? "beacon"))
        .WithTracing(x => x
            .AddAspNetCoreInstrumentation(y => y.Filter = z => !z.Request.Path.StartsWithSegments("/beacon/api/health"))
            .AddHttpClientInstrumentation()
            .AddBeaconInstrumentation()
            .AddBeaconMcpSdkTracing()
            .AddOtlpExporter())
        .WithMetrics(x => x
            .AddAspNetCoreInstrumentation()
            .AddHttpClientInstrumentation()
            .AddBeaconInstrumentation()
            .AddOtlpExporter());
}
```

With no `OTEL_EXPORTER_OTLP_ENDPOINT`, none of this runs: there is no OpenTelemetry registration at all. Everything else is driven by the standard `OTEL_*` environment variables Beacon never reads
itself: `OTEL_SERVICE_NAME`, `OTEL_TRACES_SAMPLER`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, and
so on. No endpoint, header or credential is ever hardcoded in Beacon.

A host that embeds Beacon (rather than running the sample host) calls `.AddBeaconInstrumentation()` (and, for the
SDK's spans, `.AddBeaconMcpSdkTracing()`) from its own tracing/metrics configuration the same way, after adding whichever OpenTelemetry SDK and exporter packages it needs.

### Trace continuity into host endpoint dispatch

`HostEndpointDispatcher` runs the dispatched host endpoint under `ExecutionContext.SuppressFlow()` (inside a fresh
`Task.Run`), so the MCP request's ambient execution context, including `Activity.Current`, does not flow into the
synthetic request. Inside the dispatched host code, `Activity.Current` is **`null`**: host `ILogger` scopes built
there carry no `TraceId`/`SpanId`, and an outbound `HttpClient` call the host code makes starts a brand-new trace of
its own rather than a child span of the MCP call.

The only link between the two traces is `HttpContext.TraceIdentifier`. Before suppressing flow, the dispatcher reads
`Activity.Current?.Id` on the still-flowing caller side and carries it in explicitly as the synthetic request's
`TraceIdentifier` (falling back to a synthetic `beacon-mcp-…` id when no activity was current). `Activity.Id` is
W3C traceparent-format, `00-<32 hex trace id>-<16 hex span id>-01`, so the 32-hex segment in the middle of
`HttpContext.TraceIdentifier` is exactly the audit row's `TraceId`. Host code that wants to correlate its own logs
or spans to the MCP call has to read `HttpContext.TraceIdentifier` and parse out that segment; it cannot rely on
`Activity.Current` being set.

## CloudWatch and collectors

**JSON console logs, no OTel needed.** Set `Logging:Console:FormatterName=json` and
`Logging:Console:FormatterOptions:IncludeScopes=true`. ASP.NET Core's `ActivityTrackingOptions` already defaults to
including `TraceId`/`SpanId`/`ParentId`, so every log line (including `Beacon.Audit` events) carries them once
scopes are on. Ship that JSON stream with `awslogs` (or any agent that reads container stdout) straight into
CloudWatch Logs; this path needs no OTel exporter at all.

**An ADOT collector**, receiving OTLP from Beacon and exporting to X-Ray/CloudWatch (or to Phoenix for local/dev
tracing), looks like:

```yaml
receivers:
  otlp:
    protocols:
      http:
      grpc:
processors:
  batch:
exporters:
  awsxray:      # or: otlp/phoenix: { endpoint: "http://phoenix:6006/v1/traces" }
  awsemf:       # CloudWatch metrics
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [awsxray]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [awsemf]
```

Point `OTEL_EXPORTER_OTLP_ENDPOINT` at the collector; the collector holds the AWS credentials or the Phoenix
endpoint, never Beacon.

## Fail-closed audit

With `Beacon:Mcp:Audit:Required = true`, `McpAuditCallToolFilter` wraps every tool call (attribute tools, saved-query
tools and host-endpoint tools alike). The tool's result is replaced with an error, *"Result withheld: the audit
record for this call could not be written. Contact your Beacon administrator."*, and a `Beacon.Audit` 9101 event is
logged when either:

- the audit row for that call failed to write, or
- the tool returned a **successful** result and no audit row was written for it at all.

An error result that was not audited passes through unchanged; it carries no data.
With `Required = false` (the default), results are always returned unchanged regardless of the audit outcome.

This is enforced **after execution**: the underlying query has already run by the time the filter withholds the
result, and the audit and signal writes are never skipped. The guarantee is *no unaudited data leaves Beacon*, not
*no unaudited execution*.

The audit row itself is written synchronously, before the filter answers, never through a queue: a tool keeps its
call on the request's `McpToolCallRecorder` (tool, caller, input, project, data source, rows or error, tables, and
for `ask`, `query` and `dry_run` only, the learning signal), and the filter writes the row and then the signal after
the tool returned or threw, never while it runs, so the row holds the call's final outcome. A tool that throws after
starting its record is recorded as failed and answers the same caller-safe error the tools' own error handling gives
(a database error's text through the database error redactor while PII detection is on); a protocol error is written
first and then answered as the protocol error it is. A tool that wrote its own row is never given a second one (its
signal still is), and one whose own write failed is written again. A tool that throws before any row was written,
including a call to an unknown tool, still gets one (tool, caller and the error, the error's text only when the
call's project is known and its content retention keeps it), its exception unchanged.

The Playground and the "run tool" REST path (`POST /beacon/api/mcp/tools/run`, `RunMcpToolHandler` →
`McpPlaygroundService`) call tools directly, not through the SDK's request pipeline, so they run them through the same
audit path (`McpAuditCallToolFilter.InvokeAsync`) against their own request scope: the same rows, and with
`Required = true` the same withheld error and 9101 event under the same conditions. A playground call refused before
any tool ran is recorded through the same recorder.

## Keeping audit content: the RequireContent lock

Whether the audit keeps question and SQL text follows each project's content retention (`RetainQueryContent`, see
[Content retention](https://moberghr.github.io/beacon/mcp/server/#content-retention)), which an Admin or SecurityAdmin can switch off on the MCP
Settings page. A deployment that must keep that evidence sets `Beacon:Mcp:Audit:RequireContent: true`: it pins
`RetainQueryContent` on, globally and for every project, like the other
[deployment locks](https://moberghr.github.io/beacon/getting-started/configuration/#deployment-locks).

- A save that would switch content retention off, globally or as a project override, is refused with `409`
  (`/errors/setting-locked`, naming `Audit:RequireContent`), an Admin's too. The MCP Settings page shows the switch
  disabled with the reason, and `GET /beacon/api/mcp/settings/locks` lists the lock in `locks`
  (`{ "fieldName": "RetainQueryContent", "lockName": "Audit:RequireContent" }`).
- A stored value it overrides (retention switched off before the lock) is kept, has no effect, and is named in the
  startup warning about overridden settings.
- `RequireContent` and `Beacon:Mcp:ForceNoContentRetention` pin the same setting in opposite directions: with both on,
  the host fails to start.

With the lock on, every audited call keeps its question and the SQL of every statement it ran until the audit retention
purges them. Plan the privacy review for that, and set `Beacon:Mcp:Audit:RetentionDays` accordingly. When a project's
retention setting cannot be read while the lock is on, the call keeps its content (logged at Error); without the lock it
is audited structurally, as before.

The audit rows and their statements are [sealed](#sealed-rows): an edit made at the database is found by the next
scheduled verification, as in the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/#what-the-trail-does-and-does-not-prove);
a deleted row is found by the row count once a run has checked it. Ship the `Beacon.Audit` events to a store you control for a copy the database cannot change.

## Retention

Audit rows are kept forever by default. Set `Beacon:Mcp:Audit:RetentionDays` to a positive number (`365` is
recommended) to purge rows older than that window. The purge itself is `IMcpAuditRetentionService.PurgeExpiredAsync`
(`src/Beacon.Core/Services/Retention/McpAuditRetentionService.cs`), a no-op while `RetentionDays` is `null` or `0`,
otherwise one `ExecuteDeleteAsync` plus a `Beacon.Audit` 9102 event with the deleted count and cutoff. A row's statements
are deleted with it.

The sample host schedules it as the Warp recurring job `mcp-audit-retention` (`30 3 * * *`,
`PurgeExpiredMcpAuditLogsJob` / `PurgeExpiredMcpAuditLogsJobHandler` in
`Beacon.SampleProject/Warp/Jobs/McpMaintenanceJobs.cs`). A host embedding Beacon schedules this itself, for example
with Warp:

```csharp
await recurringJobPublisher.AddOrUpdateRecurringJob(new PurgeExpiredMcpAuditLogsJob(), "mcp-audit-retention", "30 3 * * *");
```

or by calling `IMcpAuditRetentionService.PurgeExpiredAsync(cancellationToken)` from its own scheduler.

## Reading the audit

`GET /beacon/api/mcp/audit` is a paged JSON export for an Auditor or an Admin (the
[Auditor permission](https://moberghr.github.io/beacon/security/authorization/#functional-permissions)). Rows come back exactly as stored; any retention
redaction was already applied at write time, so this endpoint never re-reads or reconstructs content. The rows are not
filtered by project membership: an Auditor reads every project's tool calls, including the SQL and question text the
retention settings keep, though the Auditor role grants no other project data. Reading the audit is itself audited:
each read is recorded first in the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/) (`audit.mcp_audit_read`, with the
filters' shape and the page; an identical read by the same actor within 60 seconds once), and fails when that record
cannot be written; 9103 (above) follows on the log stream.

| Query parameter | Required | Notes |
|---|---|---|
| `from`, `to` | Yes | UTC. A value without an offset is taken as UTC; a value with an offset is converted to UTC. The range is validated after that normalisation: `to` must not be earlier than `from`, and the span must not exceed 93 days. |
| `projectId`, `tool`, `callerHash`, `userId` | No | Exact-match filters. |
| `page` | No (default `1`) | Must be `≥ 1`. |
| `pageSize` | No (default `100`) | Must be between `1` and `500`. |

Each row carries its `contentHash` and its `statements` (`id`, `purpose`, `dataSourceId`, `rowCount`, `durationMs`,
`errorClass`, `sqlHash`, and `sql`, which is `null` unless the call's project retained query content), in the order they
ran.

Any other caller gets `403`. An invalid range or paging parameter gets `400`. The **MCP Audit** page
(`/beacon/mcp-audit`, shown to Auditors and Admins) reads it, and the generated TypeScript client exposes it as
`getMcpAuditLogs`.
Security-relevant actions other than tool calls (sign-ins, users, keys, settings and the rest) are in the separate
[security audit trail](https://moberghr.github.io/beacon/security/audit-trail/).


---

# MCP gateway for your application

> Give AI clients (Claude, Claude Desktop, Cowork, Claude Code) governed access to your own ASP.NET Core application: your users sign in with Entra, your endpoints become tools that run as them, your docs become grounding. One package, about thirty lines, no data sources.

Source: https://moberghr.github.io/beacon/mcp/gateway/

This page is the whole recipe for one use case: **an MCP server for your own application.** People connect Claude to
`https://your-app/beacon/mcp`, sign in with Microsoft Entra, and Claude calls a set of your existing endpoints **as that
person**, through your own authorization, with every call audited. Beacon's data product (data sources, SQL, saved queries,
subscriptions) is not part of it. If that is what you want, follow [Integrate Beacon](https://moberghr.github.io/beacon/integrate/) instead.

:::tip[Using Claude Code or another coding agent?]
With Claude Code, install Beacon's plugin: `/plugin marketplace add moberghr/beacon`, then
`/plugin install beacon@moberg-beacon`. Its skills (`beacon-gateway`, `beacon-tool`, `beacon-entra`) follow this guide.
With any agent, point it at this guide in one file:

```text wrap
Add an MCP gateway to this ASP.NET Core app with Beacon.
Read https://moberghr.github.io/beacon/llms-gateway.txt first and follow it step by step.
Ask me before every decision it marks as one.
```
:::

## Decisions to make first

An agent should ask these, not guess.

| Decision | Options | Default |
|---|---|---|
| Who may connect | Every active user of your application, or a subset | Every active user your users table knows |
| What a tool call may do | Self-service only (the person's own records), or what the person may do in the browser | **Self-service only**: an agent reads other people's free text and sends what it reads to a model |
| Which endpoints become tools | Any endpoint with authorization metadata | Reads of the person's own data, plus writes the person would do themselves |
| Admin UI for the audit and settings | On (behind one of your policies) or off | On |
| Beacon's database | PostgreSQL (with pgvector available) or SQL Server; the app's database in schema `beacon`, or a separate one | The app's own database, schema `beacon` |

## 1. Packages and secrets

```bash
dotnet add package Moberg.Beacon.Gateway
dotnet add package Moberg.Beacon.Core.PostgreSql      # or Moberg.Beacon.Core.SqlServer
dotnet add package Moberg.Beacon.Testing              # in the test project
```

`Beacon:EncryptionKey` is required: base64 of 32 random bytes (`openssl rand -base64 32`), from user secrets or your secret
store, never a committed file. It keys the caller hashes in the audit. Set `Beacon:PublicBaseUrl` to the URL clients use
(`https://your-app`): the discovery documents and challenges advertise `<PublicBaseUrl>/beacon/mcp`.

## 2. Who a token is: `IMcpHostUserResolver`

Beacon validates the Entra token (tenant, audience, delegated, scope, client). Your application decides who the person
is and what a tool call carries:

```csharp
internal sealed class AppMcpUsers(AppDbContext db) : IMcpHostUserResolver
{
    public async Task<McpHostUser?> FindAsync(McpTokenIdentity token, CancellationToken ct)
    {
        var users = await db.Users
            .Where(x => x.ObjectId == token.ObjectId)
            .Where(x => x.IsActive)
            .Select(x => new { x.Id, x.DisplayName })
            .Take(2)
            .ToListAsync(ct);

        // Exactly one, or nobody: an object id two active users carry does not say which of them this is.
        if (users.Count != 1)
        {
            return null;
        }

        return new McpHostUser(
            users[0].Id.ToString(),
            users[0].DisplayName,
            [new Claim("app_user_id", users[0].Id.ToString())]);   // the claims your own sign-in adds
    }
}
```

- Look users up by `ObjectId` (`oid`), never by `sub`: the token was issued to another client, so its `sub` differs.
- Never create or link a user here. Someone who has never signed in to your application is refused.
- It runs on every tool call, so deactivating a person in your application cuts MCP off at their next call.

## 3. Program.cs

```csharp
// Register any IClaimsTransformation of your own BEFORE the gateway (it runs yours first).
builder.Services.AddBeaconMcpGateway(builder.Configuration, x =>
{
    x.ProjectName = "app";                                                        // Beacon's project for your tools and docs
    x.Database(db => db.UsePostgreSql(sp => sp.GetRequiredService<NpgsqlDataSource>()));  // or db.UsePostgreSql() with ConnectionStrings:BeaconContext
    x.Entra(builder.Configuration.GetSection("AzureAd"))                          // TenantId, ClientId (Microsoft.Identity.Web's section)
        .AllowClients(builder.Configuration.GetSection("McpAccess:AllowedClientIds"));  // optional: only Claude's client
    x.Users<AppMcpUsers>();
    x.Tools(y =>
    {
        y.AllowWriteTools = true;                                                 // only if you mark write tools
        y.BindFromCaller("userId", "app_user_id");                                // the person's own id, never the agent's
    });
    x.Docs(y => y.FromEmbeddedResources(typeof(Program).Assembly, "app-docs/"));   // optional grounding
    x.AdminUi(y =>                                                                // optional
    {
        y.HostPolicy = "beacon.admin";                                            // one of YOUR authorization policies
        y.HostUserId = z => z.FindFirst("app_user_id")?.Value;
        y.SignInPath = "/Account/SignIn";
    });
});

var app = builder.Build();
app.UseBeaconMcpGateway();          // before UseAuthentication
// ... UseStaticFiles (the admin UI's assets), UseAuthentication, UseAuthorization ...
app.MapBeaconMcpGateway();          // /beacon/mcp, discovery documents, the admin UI
```

`UseBeaconMcpGateway()` also adds Beacon's security headers and its Host and Origin check on `/beacon` (an OpenID
Connect callback always passes the check), as `UseBeaconAuthentication()` does for a full host. See
[Security headers and the origin check](https://moberghr.github.io/beacon/integrate/host-setup/#security-headers-and-the-origin-check).

What you get, and what you do not have to write:

- **Callers:** delegated tokens of your tenant and registration with the `mcp.access` scope (`RequireScope` changes it),
  whose person your resolver knows. App-only tokens, ID tokens, other tenants and other scopes are refused. Entra roles and
  groups are never read.
- **Tools:** your marked endpoints as `api_<name>`, plus `get_context`, `search` and `get_documentation` for your docs. No
  SQL tools, no saved-query tools, no data sources. Each tool's description ends with a note that it runs with the
  caller's permissions in your application; when your endpoints run with less (self-service only, say), word it yourself
  with `y.DescriptionNote` in `x.Tools(...)` (see [Host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/)).
- **Locks:** read-only and no content retention are pinned in code (`RetainToolContent = true` keeps call content).
- **Your session never counts on `/beacon/mcp`,** and a bearer token never counts on your routes. The MCP route answers
  401/403 itself, never your sign-in redirect: you edit no cookie events.
- **Discovery:** `/.well-known/oauth-protected-resource` publishes your tenant's v2 issuer and
  `api://<ClientId>/mcp.access` plus `offline_access` (so clients refresh instead of re-prompting hourly). It names the
  server `Beacon MCP` (the server card and `initialize`: `Beacon`) and links Beacon's docs unless you set
  `Beacon:Mcp:ProtectedResource:ResourceName` and `ResourceDocumentation` (an https URL) to your application's. The
  server card describes the gateway (your endpoints and documents); `x.ServerDescription` words it yourself.
- **The host project** is synced the first time it is needed; the `beacon-mcp-gateway` health check reports Degraded until
  then. No hosted service, no startup step.
- **Admin UI** at `/beacon`: the MCP audit, the MCP settings, the learning views and your project, for holders of
  `HostPolicy`, signed in with your own sign-in. Every other Beacon endpoint answers 404.

## 4. Migrations

Beacon's tables live in schema `beacon`. Apply them from your migration step, not from every web replica:

```csharp
await Beacon.Core.PostgreSql.BeaconPostgreSqlMigrator.MigrateAsync(dataSource);   // or (connectionString)
await Beacon.Core.SqlServer.BeaconSqlServerMigrator.MigrateAsync(connectionString);
```

On PostgreSQL one migration creates the `vector` extension: allow it (Azure Database for PostgreSQL: add `VECTOR` to
`azure.extensions`, which replaces the whole list). Sharing your `NpgsqlDataSource` needs connection reset on and
multiplexing off; Beacon refuses one that does either.

## 5. Mark endpoints as tools

On a minimal API endpoint (`using Beacon.MCP.HostEndpoints;` for `WithBeaconTool`), a controller action, or a handler class
whose framework copies class attributes into the endpoint (a Moberg Warp handler; `using Beacon.Core.HostEndpoints;` for
`[BeaconTool]`):

```csharp
app.MapGet("/api/my/orders", (int userId, OrderService orders) => orders.Of(userId))
    .RequireAuthorization()
    .WithBeaconTool("my_orders", "The signed-in user's orders, newest first: id, placedAt (UTC), status, total.", readOnly: true);

[Authorize]
[BeaconTool("request_leave", Description = "...", ReadOnly = false, Destructive = false, Idempotent = false)]
[WarpHttpPost("/api/time-off/requests")]
public sealed class RequestLeaveCommandHandler(...) { ... }
```

The rules an agent follows for every tool:

1. **Self-service.** A tool reads or changes the person's own records. Never mark an admin endpoint or one gated by a
   permission a person might hold over others.
2. **Data minimisation.** If an endpoint returns other people's data to some callers (a manager's view, a list by
   sequential id), do not mark it: add a `/api/my/...` sibling that filters on the caller and takes no subject.
3. **Authorization metadata.** The endpoint needs `[Authorize]` or a policy; startup refuses one without (unless you have a
   fallback policy). The call runs through the endpoint's own authorization as the user your resolver returned.
4. **The person's id is bound, not asked.** `BindFromCaller("userId", "app_user_id")` takes `userId` out of the tool's
   schema and fills it from the caller; an agent sending it is refused. A name no tool takes is warned at startup.
5. **Descriptions are for the agent:** what comes back, field by field; what each argument means; every refusal; for a
   write tool, "call it only when the person asked for this change". Beacon appends the route and the read/write flag.
6. **`ReadOnly` is always declared.** A write tool (`ReadOnly = false`) needs `AllowWriteTools`, is offered to signed-in
   people only, makes clients ask before each call, and is audited as `api_<name>:write`.

### Recognising an agent call

A tool call's principal has the authentication type `BeaconMcpUser`. Use it to hold agents to less than the browser —
self-service only, whatever the person may do there:

```csharp
if (httpContext.User.IsBeaconMcpCall())
{
    // no permissions, no manager grants: the person's own records only
}
```

### Refusals reach the agent as your problem details

Tool calls run your `IExceptionHandler`s, as `UseExceptionHandler()` does for the browser: a refusal your application maps
to a 403/404/400 problem reaches the agent as that status and body. An exception no handler takes is a generic error.

## 6. Entra app registration

On **your application's existing registration** (the `AzureAd:ClientId` one):

- [ ] Manifest: `api.requestedAccessTokenVersion = 2`. First — Entra only accepts an HTTPS Application ID URI on a v2 app.
- [ ] Expose an API: keep `api://<clientId>` and add the exact MCP URL `https://<your-domain>/beacon/mcp` to
      `identifierUris`. Claude sends it as the `resource`; a mismatch fails with **`AADSTS9010010`**. The domain must be a
      verified domain of the tenant (`*.azurecontainerapps.io` and similar cannot be).
- [ ] Expose an API: add the delegated scope **`mcp.access`**.

**A client registration for Claude** (Entra has no dynamic client registration):

- [ ] New app registration, Web platform, redirect URI `https://claude.ai/api/mcp/auth_callback`.
- [ ] A client secret (record its expiry: the connector stops when it lapses).
- [ ] API permissions: your API's `mcp.access` (delegated) and `offline_access`; grant admin consent.
- [ ] Give the client id and secret to whoever adds the connector in Claude; optionally list the client id in
      `McpAccess:AllowedClientIds`.

## 7. Test it

```csharp
var tokens = new McpTestTokens(tenantId, clientId);
var users = new McpTestUsers().Add(objectId: "oid-1", id: "42", displayName: "Ada", new Claim("app_user_id", "42"));

builder.Services.TrustMcpTestTokens(tokens);   // after AddBeaconMcpGateway, in the test host only
builder.Services.UseMcpTestUsers(users);

await using var client = await McpTestClient.ConnectAsync(app.GetTestClient(), tokens.For("oid-1"));
var tools = await client.ListToolsAsync();
var result = await client.CallToolAsync("api_my_orders");
```

For trying the gateway by hand on a laptop without Entra, run a local OIDC mock (for example
`ghcr.io/navikt/mock-oauth2-server`) that mints delegated tokens with your tenant id, audience, an `oid` and
`scp=mcp.access`, and point `AzureAd:Instance`/`TenantId` validation at it in Development only.

The client is the official MCP SDK's, so a test goes through the bearer middleware, the caller mapping, the route policy,
the MCP filters and audit, and your endpoint's own authorization. Listing and calling tools needs Beacon's database (project
access and the audit live there): run those tests against a real PostgreSQL or SQL Server with Beacon's schema migrated.

Pin, in your own tests: exactly which endpoints are tools (and which write), that each refuses somebody else's data, and
that an agent call is self-service only.

## 8. Verify a deployment

`beacon-doctor` walks the sign-in path the way an MCP client does and prints PASS/WARN/FAIL (exit code 1 on a failure):

```bash
dotnet tool install -g Moberg.Beacon.Doctor
beacon-doctor mcp https://your-app/beacon/mcp                    # the challenge, discovery, Entra metadata, absent documents
beacon-doctor mcp https://your-app/beacon/mcp --token "$TOKEN"   # plus initialize and tools/list as that person
```

By hand:

- [ ] `POST /beacon/mcp` without a token → 401 with `WWW-Authenticate: Bearer resource_metadata=...`.
- [ ] `/.well-known/oauth-protected-resource` → your v2 issuer, `api://<clientId>/mcp.access`, `offline_access`, and a
      `resource` equal, character for character, to an `identifierUris` entry.
- [ ] `/health` → `beacon-mcp-gateway` Healthy.
- [ ] Claude connects, lists your tools, a read answers, and a write tool asks before it runs.
- [ ] After an hour the connector refreshes silently.

| Symptom | Cause |
|---|---|
| `AADSTS9010010` | The MCP URL is not an `identifierUris` entry (exactly), or the domain is not verified. |
| 401 `invalid_token` | Wrong issuer or audience: check `AzureAd:TenantId` / `ClientId`, and that the token is v2. |
| 403 at `/beacon/mcp` | The token is valid but not a caller: wrong scope or client, the person unknown to your resolver, or the host project not synced yet. The log has the reason with the caller hash. |
| 503 with `Retry-After` | Entra's signing keys could not be fetched. |
| Re-prompted to sign in every hour | `offline_access` missing from `scopes_supported` or not consented. |
| A tool error "failed with an unexpected error" | The endpoint threw and no `IExceptionHandler` took it. |
| Startup: "relies on '…' being its only authentication scheme" | Name your default: `AddAuthentication("YourScheme")`. |
| Startup: "Register your IClaimsTransformation before calling AddBeaconMcpGateway" | Move your registration above the gateway's. |


---

# Saved Query Tools

> Expose approved, parameterized saved queries as named, read-only MCP tools such as q_loan_book_by_month(from, to), so agents and routines call reviewed, versioned SQL instead of carrying their own.

Source: https://moberghr.github.io/beacon/mcp/saved-query-tools/

An agent that writes its own SQL for a recurring question gets a slightly different query every time. A **saved query
tool** turns a reviewed Beacon query into a fixed MCP tool, such as `q_loan_book_by_month(from, to)`, that always runs the
same approved SQL with the caller's arguments bound as database parameters. Routines become deterministic and
cheaper: they call a tool instead of shipping `queries/*.sql` files or asking `ask` to regenerate SQL.

## Expose a query

1. Write the query as usual. Parameters are `{name}` placeholders in the step SQL, with a type (Number, DateTime,
   String) and a description. The description becomes the argument's description in the tool schema.
   A placeholder is `{identifier}` (letters, digits, underscore, starting with a letter or underscore); newly exposing a
   query whose placeholder has another shape is refused with the parameter's name, and editing the query to a valid
   placeholder fixes it. A tool that is already exposed keeps running and can be edited or disabled as stored.
2. Get a version **approved** (see [the approval rule](#which-version-runs)). With the approval workflow enabled,
   saving SQL changes submits a version for approval; an Approver (or an Admin) other than the requester approves it on
   **Approvals** (`/beacon/approvals`; see [Queries → Approval workflow](https://moberghr.github.io/beacon/features/queries/#approval-workflow)).
3. On the query's detail page, a **SecurityAdmin** (or an Admin) opens the **MCP tool** card, ticks *Expose as MCP
   tool*, sets a name and (optionally) a description, and saves. The card shows which approved version calls run, or why
   none can. While the approval workflow is on, publishing is a third pair of eyes: whoever requested or approved the
   version the tool runs cannot enable the tool, nor can a caller Beacon cannot name (an API key without an active
   owner), who could be either (`403`), unless the host allows self-approval (`ApprovalWorkflow.AllowSelfApproval`).
   The check is made when the tool is enabled: once published, the tool serves whichever version is approved next,
   whoever requested or approved that one. With the workflow off nobody approves a change, and enabling is not checked
   against approvals.

The same through the API (the SecurityAdmin policy, see [Functional permissions](https://moberghr.github.io/beacon/security/authorization/#functional-permissions); anyone but an Admin publishes only a query they can read, `404` otherwise):

```http
PUT /beacon/api/queries/42/mcp-tool
Content-Type: application/json

{ "enabled": true, "name": "loan_book_by_month", "description": "Outstanding loan book per month for a date range." }
```

Rules the endpoint enforces:

| Field | Rule |
| --- | --- |
| `name` | `^[a-z][a-z0-9_]{2,40}$`; the MCP tool is `q_<name>`. Unique across live (non-archived) queries, so it is unique in every project and always names one query. Required to enable. Two concurrent saves of the same name get the same "already used" error (`400`), never a `500`. |
| `description` | Optional, at most 1000 characters. Falls back to the approved version's description, then its name. |
| `enabled: true` | Requires a runnable approved version whose parameters make a valid tool (see below), and, while the approval workflow is on, a signed-in caller with a user id who neither requested nor approved that version (`403` otherwise, unless the host allows self-approval; checked when enabling, not again when a later version is approved). Disabling is always allowed, by anyone who may publish; the name stays reserved while it is set. Disabling with a name another query already uses succeeds and keeps the query's stored name. |

The query detail DTO (`GET /beacon/api/queries/{id}`) carries `mcpToolEnabled`, `mcpToolName`,
`mcpToolDescription`, `mcpToolRunnableVersionNumber` and `mcpToolIssue`. `mcpToolIssue` explains why the query is not
a tool anyone can call: no approved active version, a parameter shape that cannot be a tool, or **not visible in any
project**: no single project contains all of the data sources its steps read, so no caller can ever see it. The
last one does not block enabling (a data source may be added to a project later); it is reported so the tool does
not silently stay invisible.

## Which version runs

A tool runs **only** the query's active version, and only when an approval request for **exactly that version** was
approved. In the workflow's strict mode (`ApprovalWorkflow.RequireApprovedVersionToRun`) the approval must also have
recorded the version's content hash, which binds it to exactly the content the reviewer approved: an approval recorded
before Beacon kept content hashes names the version but not what it approved, so strict mode does not run that tool until
the version is approved again (restore it and approve the submission). Outside strict mode such an
approval still counts, and with the workflow on the host logs a startup Warning naming those tools (at most 20, with
their count). In strict mode the startup Warning counts instead the queries with a tool or a subscription whose active
version is not approved for its content hash. Nothing else is callable:

| Version state | Callable? |
| --- | --- |
| Active, its approval request Approved (in strict mode: for the version's content hash) | **Yes**. This is the only runnable version |
| Active, approved before Beacon kept content hashes | Outside strict mode only (named in the startup Warning) |
| Pending approval (a later edit) | No. The tool keeps running the approved version until the edit is approved |
| Rejected, or superseded by a newer submission | No |
| Active without an approval (edited while the approval workflow was off, or restored from history while it was off) | No |
| Draft / no version | No |

A query with no runnable version is **not listed** as a tool at all, even when enabled. The tool executes the
approved version's **snapshot** (its stored steps and final query), not the live step rows, so an unreviewed change
to the live query can never change what the tool runs. In practice: enable the approval workflow on hosts that use
saved-query tools. It is off by default; turn it on in code with `options.EnableApprovalWorkflow(...)` in
`AddBeaconServices` (see [Approvals](https://moberghr.github.io/beacon/security/approvals/)). With the workflow on, a version restore is
submitted for approval like an edit, so restoring an older version never makes it callable without a new approval.
Subscriptions follow the same rule in the workflow's strict mode
(`ApprovalWorkflow.RequireApprovedVersionToRun`, see [Queries → Approval workflow](https://moberghr.github.io/beacon/features/queries/#approval-workflow)).

A version is also not callable when its parameters cannot form a tool: a parameter without a placeholder, a name
that is not a plain identifier, the reserved name `project_id`, or the same parameter declared with different types
in different steps. The MCP tool card shows the reason.

## What the agent sees

Each tool is listed with:

- **name** `q_<name>`, title (the query name) and the description, plus the approved version number;
- an **input schema** built from the parameters: `number` for Number, an ISO 8601 `string` for DateTime, a
  `string` (max 4000 characters) for String; every parameter is `required` (the query model has no defaults) and
  `additionalProperties` is false;
- annotations `readOnlyHint: true`, `destructiveHint: false`, `idempotentHint: true`, `openWorldHint: false`.

Arguments are validated before anything runs. A missing or unknown argument, a non-number for a Number, or a date
that is not ISO 8601 is a tool error. Dates with `Z` or an offset are normalized to UTC; a bare date
(`2026-01-31`) is passed as written.

The result is a markdown table plus structured content:

```json
{
  "columns": [{ "name": "month", "type": "string" }, { "name": "principal", "type": "number" }],
  "rows": [{ "month": "2026-01", "principal": 1250000.00 }],
  "rowCount": 1,
  "truncated": false,
  "queryVersionId": 318,
  "versionNumber": 4
}
```

`truncated` is true when the result had more rows than the project's MCP row limit (or rows were dropped to keep the
payload within 256 KB, then `rowsOmittedForSize` is also true).

**Many tools.** Up to `SavedQueryTools.NamedToolLimit` tools (default 25) visible to one caller, each is its own
`q_<name>` tool. Above it, the caller gets two catalog tools instead:

| Tool | Parameters | Returns |
| --- | --- | --- |
| `search_saved_queries` | `query` (string, keywords matched against names and descriptions; empty lists every tool), `limit` (integer, default 10, max 50) | names, descriptions and input schemas |
| `run_saved_query` | `name` (string, required), `arguments` (object matching the query's input schema, including `project_id` when the schema lists one) | the same result as the named tool |

Configure the limit in `AddBeaconServices(..., options => options.SavedQueryTools.NamedToolLimit = 40)`.

The project brief (`get_context` with `format: "agents_md"`) lists the project's saved-query tools with their
parameters, so an agent workspace's `AGENTS.md` knows they exist.

The tools coexist with the built-in tools and with [host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/)
(`api_<name>`): all three are served side by side.

## Using one from a routine

An agent routine that refreshes a dashboard's data calls the tool and writes the structured rows as JSON, with no SQL
in the routine. The `mcp.callTool` call below stands for whatever MCP client the routine runs in:

```ts
// routines/refresh-loan-book.ts

const result = await mcp.callTool("beacon", "q_loan_book_by_month", {
  from: "2025-10-01",
  to: "2026-10-01",
});

if (result.isError) {
  throw new Error(result.content[0].text);
}

const { rows, truncated, versionNumber } = result.structuredContent;
if (truncated) {
  throw new Error("Loan book result was truncated; narrow the date range or raise the project's row limit.");
}

await writeFile(
  "dist/data/loan-book.json",
  JSON.stringify({ generatedFrom: `q_loan_book_by_month v${versionNumber}`, rows }, null, 2),
);
```

When the approved SQL changes, the routine picks up the new version on its next run without an edit, and the
`versionNumber` it records says which reviewed SQL produced the file.

## Security model

- **Authorization is the caller's projects.** A tool is visible in a project only when that project contains
  **every** data source the approved version reads, and only in the caller's projects where its
  [project access](https://moberghr.github.io/beacon/security/authorization/#project-access) lets it run saved SQL (`Run`; the projects an API key or
  mapped caller reaches are described in the [MCP Server guide](https://moberghr.github.io/beacon/mcp/server/#overview)): a query over a
  data source shared by projects A and B shows in both only if each contains all of its sources, and a caller of A
  never sees a query that needs a B-only source. A tool outside your projects answers exactly like one that does not
  exist. A caller with no project access sees no tools (fail closed). When your credentials reach the same tool
  through several projects, the schema offers `project_id` and the call requires it.
- **Visibility follows data-source membership, not an owning project.** A saved query has no owning project: a tool
  appears in every project of the caller that contains all of its data sources, so linking a data source to another
  project can make an existing tool visible there. The tool's SQL is never listed, and the caller's `Run` permission
  is still checked in each project and again right before the query runs.
- **Re-checked before it runs.** Right before any SQL runs, the call checks the steps it is about to run again: the
  caller may still run saved SQL in the project, and the project still links every data source those steps read.
  Otherwise the call fails ("not available in project N for your credentials") without running anything, and is
  audited.
- **Execute scope.** `/beacon/mcp` requires the Execute scope, so a Read-scoped API key cannot call saved-query
  tools; project-restricted API keys work and are held to their project list.
- **Read-only execution.** Every step's SQL, with its arguments bound, goes through the shared execution gate
  (regex guardrail, AST read-only validator, cross-database rule, system catalog rule, row limit) and runs through the provider's read-only
  path: a `READ ONLY` transaction on PostgreSQL and MySQL; on the other engines the parser gates and the read-only
  login you connect with (see [How a query runs](https://moberghr.github.io/beacon/features/queries/#how-a-query-runs)). A final query joins the steps
  in in-memory SQLite behind the same gate.
- **Parameters, never interpolation.** Each `{placeholder}` is replaced by a generated database parameter
  (`@p0`, `@p1`, …) and the value is sent as a parameter; argument text never reaches the SQL.
- **Host-managed sources.** When a step reads a [host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/) data source, the host's
  allow-list and column policy apply, in the gate and again in the provider, and masked columns come back masked.
- **Row limits and PII.** The final result is capped at the project's MCP row limit. An intermediate step of a
  multi-step query may return at most 50,000 rows; past that the call fails instead of joining an incomplete set.
  PII columns are masked per the project's PII detection setting before rows leave a step.
- **Audit.** Every call, whether it succeeds or fails and including unknown tools and invalid arguments, is written to the MCP
  audit log as `q_<name>` (a search as `search_saved_queries`), with project, data source, row count and error.
  Arguments are kept only when the project's retention settings allow content, and are never written to the
  application log. The project is resolved **before** the arguments are validated, so an invalid-argument row
  follows that project's retention setting. When no project can be established (an ambiguous or foreign
  `project_id`, an unknown tool, a search across several projects) the row is structural only (tool, input size and
  error class, never argument content) rather than falling back to the global setting.
- **No learning signals.** Saved queries are pre-approved SQL, not the caller's SQL, so they are **audit-only**:
  `McpSignalService` does not record them and they never feed pattern mining or golden examples.

## Known limits

- Parameters have no defaults or allowed-value lists in the query model, so every argument is required and free-form
  within its type.
- Only database data sources run as tools; API data source steps are rejected.
- A multi-step query whose intermediate step returns no rows cannot join (the in-memory table is not created); the
  call fails with the SQLite error.
- Column types in the structured result are inferred from the values returned; an empty result has no columns.


---

# Host Endpoint Tools

> Expose selected endpoints of the host application as MCP tools (read-only by default, write tools behind an explicit opt-in) invoked in-process as the caller through the endpoint's own model binding, filters and authorization policies.

Source: https://moberghr.github.io/beacon/mcp/host-endpoint-tools/

A host application already has endpoints that answer the questions an agent asks: a back-office grid of a
customer's loans, a lookup by id. `AddHostEndpointTools` turns the endpoints you mark into MCP tools. Each call runs
**in-process, through the endpoint's real pipeline** (model binding, action filters, authorization policies), as a
host identity built for the MCP caller, never as Beacon, and never with the caller's own token.

Every exposed endpoint **declares** whether it is read-only; Beacon never infers it from the HTTP verb, because
back offices read over POST (Kendo grids, `[FromForm]` filters). A read-only endpoint on any verb other than GET or
HEAD is named in a startup Warning; set `AllowReadOnlyPost = false` to expose read-only tools on GET and HEAD only.
By default only read-only endpoints are exposed; endpoints that
change data become tools only when the host opts in with `AllowWriteTools` (see [Write tools](#write-tools)).

## Quick start

```csharp
builder.Services.AddBeaconMcp()
    .AddHostEndpointTools(o =>
    {
        o.ProjectName = "Back Office";                 // the tools belong to this Beacon project
        o.MaxResponseBytes = 1_048_576;                // default 1 MiB
        o.NamedToolLimit = 20;                         // above this: search_api + call_api
        o.RequestTimeout = TimeSpan.FromSeconds(30);   // default 30 s
        o.MaxConcurrentCalls = 64;                     // default 64 calls in flight in total
        o.MaxConcurrentCallsPerCaller = 16;            // default 16 per MCP caller
        o.QueueWait = TimeSpan.FromSeconds(5);         // how long a call over a limit waits before it is refused
        // o.AllowReadOnlyPost = false;              // default true; false: read-only tools on GET/HEAD only
        // o.TrustedAuthenticationSchemes = ["..."];   // default none; see "Authentication schemes" below
        // o.AllowWriteTools = true;                   // default false; see "Write tools" below
        // o.DescriptionNote = tool => "...";          // default Beacon's note; see "Description" below
    });
```

`ProjectName` names the **host project**, the same one `ExposeDbContext` and `ExposeDocs` use. Beacon finds it by
its host key (`Project.HostManagedKey = host:{name, lower-cased}`), never by its display name, so a project a Beacon
user creates with the same name never gets the tools (see [Host documentation](https://moberghr.github.io/beacon/mcp/host-docs/#the-host-project)).
The tools appear once the host has run `SyncBeaconHostAsync` and the project exists.

Mark a controller action:

```csharp
[HttpPost]
[Authorize]                                           // required: [Permission] is a filter, not authorization metadata
[Permission(AdminPermission.ViewLoans)]               // the host's own filter: runs in the action pipeline (needs a dispatch middleware, below)
[BeaconTool("customer_loans", Description = "Loans of one customer, newest first.", ReadOnly = true)]
public IActionResult CustomerLoans([DataSourceRequest] DataSourceRequest request, int customerId)
{
    // ...
    return Json(loans.ToDataSourceResult(request));
}
```

Or a minimal API endpoint:

```csharp
app.MapGet("/api/loans/{id:int}", (int id, LoanService loans) => loans.Find(id))
    .RequireAuthorization("ViewLoans")
    .WithBeaconTool("loan_by_id", "One loan by id.", readOnly: true);
```

Or a handler class whose framework copies class attributes into its endpoint's metadata, such as a
[Moberg Warp](https://moberghr.github.io/warp/) handler with a `[WarpHttp*]` route:

```csharp
[Authorize]
[BeaconTool("my_assets", Description = "The signed-in user's assets, newest first.", ReadOnly = true)]
[WarpHttpGet("/api/my/assets")]
public sealed class ListMyAssetsQueryHandler(AppDbContext db) : IRequestHandler<ListMyAssetsQuery, IReadOnlyList<AssetListItem>>
{
    // ...
}
```

On an MVC controller class `[BeaconTool]` is refused at startup: it would mark every action under one name, so it goes
on the action.

The tool is exposed as `api_customer_loans` (or `api_loan_by_id`, `api_my_assets`).

The endpoint must carry authorization metadata (`[Authorize]` on the action or controller, a policy, or
`[AllowAnonymous]`), or the host must have a fallback authorization policy; otherwise startup refuses it. A
permission check implemented as a filter (like the `[Permission]` filter above) is not metadata. A host that has
neither `[Authorize]` on its controllers nor a fallback policy puts `[Authorize]` on every exposed action, next to
`[BeaconTool]` and its own permission filter, as above.

An exposed endpoint whose only requirement is an authenticated user (`[Authorize]` or `RequireAuthorization()` with
no policy or roles, under the default policy, or a fallback policy that only requires authentication) is logged as a
startup **Warning** naming it: every mapped MCP caller of the project can call it. It is still exposed, since a
permission filter like `[Permission]` may check the rest; give it a policy or roles if nothing does. A tool that binds
an argument from the caller ([Arguments the caller fills](#arguments-the-caller-fills)) is the self-service pattern
and is not warned.

### Authentication schemes

Beacon authorizes the dispatched call against the host principal it built for the MCP caller; it never runs an
authentication handler. A policy that names **authentication schemes** (`[Authorize(AuthenticationSchemes = "StepUp")]`,
`new AuthorizationPolicyBuilder("PartnerApi")`, a default or fallback policy with schemes) would therefore be
evaluated as if that scheme had authenticated the caller: a step-up or MFA scheme would be bypassed. So an exposed
endpoint whose combined policy names any scheme **fails startup**, naming the endpoint and the schemes, unless the
host lists each scheme in `TrustedAuthenticationSchemes`:

```csharp
.AddHostEndpointTools(o =>
{
    o.ProjectName = "Back Office";
    o.TrustedAuthenticationSchemes = ["Cookies"];      // only if the MCP caller is an acceptable stand-in for it
});
```

List a scheme only when an authorized MCP caller really should count as authenticated by it. Never list a step-up,
MFA or partner scheme.

## What gets exposed

- Only endpoints carrying `[BeaconTool]` / `.WithBeaconTool(...)`. Everything else is invisible.
- The name must match `^[a-z][a-z0-9_]{2,47}$` and be unique. `ReadOnly` must be declared: a missing flag **fails
  startup** with a message naming the endpoint, and so does `ReadOnly = false` unless `AllowWriteTools` is on (see
  [Write tools](#write-tools)). So do bad or duplicate names, `Destructive = true` on a read-only tool, file-upload parameters,
  `[AsParameters]`, an endpoint with no authorization at all, and an endpoint whose policy names an authentication
  scheme that is not in `TrustedAuthenticationSchemes` (see above).
- `ReadOnly = true` on a verb other than GET or HEAD (GET is used when the endpoint takes several verbs) logs a
  startup Warning naming the endpoint. With `AllowReadOnlyPost` on (the default) it is still exposed; with it off it
  is not. A read-only endpoint on PUT, PATCH or DELETE most likely changes data: declare it a write tool.
- Each tool name has one owner, checked once the pipeline is built. A name the saved-query tools (`q_<name>`,
  `search_saved_queries`, `run_saved_query`, while saved-query tools are served) and the endpoint tools (the `api_<name>`
  tools discovered, or `search_api` / `call_api` in catalog mode) both claim **fails startup**, naming it. An attribute
  tool the host adds with such a name only logs a warning: calls to that name reach the attribute tool. In `tools/list`
  the saved-query tools come before the endpoint tools (`AddBeaconMcp()` first, then `AddHostEndpointTools`).
- Endpoints are discovered when the host's pipeline is built (they do not exist at DI time), from the
  `EndpointDataSource`. Parameters come from **ApiExplorer** when it describes the endpoint (`[ApiController]`
  controllers, or `AddEndpointsApiExplorer()` for minimal APIs), otherwise from the action's parameters and binding
  attributes. Swagger / Swashbuckle is never needed.
- **Description**: the attribute's `Description`, else the action's XML doc `<summary>` (enable
  `GenerateDocumentationFile`; `<param>` docs describe the arguments), else the endpoint's summary/description
  metadata, else a generic "Calls POST /route" line. Every description ends with a note naming the method and route
  and saying the tool runs with the caller's host permissions (a write tool's also says it changes data and is to be
  called only when the user asked). When the endpoints run with less than the caller's own permissions, or you word it
  otherwise, set `DescriptionNote`: it gets each tool's name, method, route and `ReadOnly` flag and returns the note,
  `null` or an empty string ends the description at the summary, and `tool.DefaultNote` is Beacon's own text:

  ```csharp
  o.DescriptionNote = tool => tool.ReadOnly
      ? $"Self-service endpoint {tool.HttpMethod} {tool.Route}: it sees only your own records."
      : tool.DefaultNote;
  ```

### Arguments

| Bound from | Tool argument |
| --- | --- |
| Route | one argument per route value, required unless optional in the template; URL-encoded into the path |
| Query / form, simple types | one argument per value; collections as arrays (sent as repeated keys) |
| Query / form, a model (`[FromForm] LoanFilter filter`, or an unannotated model on a plain MVC controller) | flattened: one argument per property (`CustomerId`, `Period.From`, up to 3 levels); collections of objects are skipped |
| `[FromBody]` | one argument named after the parameter, its schema from `System.Text.Json`'s `JsonSchemaExporter` |
| `[FromHeader]` | one argument named after the header. `Authorization`, `Cookie`, `Host` and content headers are refused |
| A custom model binder (Kendo's `[DataSourceRequest]`) | one object argument whose members are sent as raw fields: `{ "page": 1, "pageSize": 20, "sort": "CreatedDate-desc" }` |
| `CancellationToken`, `HttpContext`, `[FromServices]`, DI-injected minimal API parameters | not arguments |

Unknown and missing required arguments are rejected before anything is dispatched.

### Arguments the caller fills

An agent does not know the person's id in your application, so an endpoint that takes it (`/api/my/assets?userId=…`, a
body with `RequestingUserId`) would otherwise need "leave it out, or pass 0, to mean me" logic. Bind such an argument to
the caller instead:

```csharp
builder.Services.AddBeaconMcp()
    .AddHostEndpointTools(x =>
    {
        x.ProjectName = "Shop";
        x.BindFromCaller("userId", "shop_user_id");           // a claim of the host principal the call runs as
        x.BindFromCaller("requestingUserId", "shop_user_id");
    });
```

On every tool, a route, query, header or form value with that name (compared ignoring case), or a top-level property of
its JSON body, is left out of the input schema and of its required list, and the call carries the caller's value — a body
property written as the type its schema declares. An agent that sends a bound name anyway — as an argument, as a raw field
of a custom-binder model, or as a body property in any casing — is refused, on every tool, since a custom binder or a
dictionary could still read it. A principal without a value refuses the call, and a bound name that no tool takes is
warned at startup (most likely a typo). A `Func<ClaimsPrincipal, string?>` overload reads the value any other way. Your
endpoint still checks ownership as it does for the browser.

### Named tools or a catalog

Up to `NamedToolLimit` tools, each is its own MCP tool. Above it, agents get two tools instead, so a large host does
not flood the tool list: `search_api` (keywords → matching tools with their input schemas) and `call_api` (a name
plus arguments). Read-only tools carry `readOnlyHint = true`, `destructiveHint = false`, `idempotentHint = true`,
`openWorldHint = false`; write tools carry the hints in [Write tools](#write-tools). `search_api` is always read-only.
`call_api` is read-only only while every tool it can reach for this caller is; once it can reach a write tool it
carries `readOnlyHint = false` (and `destructiveHint = true` if any of them is destructive), so clients confirm every
`call_api` call. Each `search_api` result carries `read_only` (and, for a write tool, `destructive` and `idempotent`).
The eight built-in tools are unaffected.

## Write tools

A host can let agents change data through its own endpoints: an HR app letting a user book or withdraw their own
leave from Claude Desktop. This is off unless the host opts in, and each endpoint still declares itself:

```csharp
.AddHostEndpointTools(o =>
{
    o.ProjectName = "Employee Portal";
    o.AllowWriteTools = true;                          // without it, ReadOnly = false fails startup
});

[HttpPost("leave")]
[Authorize(Policy = "Employee")]
[BeaconTool("book_leave", Description = "Book leave for yourself.", ReadOnly = false)]
public IActionResult BookLeave([FromBody] BookLeaveRequest request) { ... }

[HttpDelete("leave/{id:int}")]
[Authorize(Policy = "Employee")]
[BeaconTool("withdraw_leave", Description = "Withdraw one of your leave requests.",
    ReadOnly = false, Destructive = true, Idempotent = true)]
public IActionResult WithdrawLeave(int id) { ... }

// Minimal API: .WithBeaconTool("withdraw_leave", "...", readOnly: false, destructive: true, idempotent: true)
```

- **Opt-in.** `AllowWriteTools` (default `false`) is the host-level switch. `ReadOnly` must still be declared on every
  endpoint; Beacon never infers it from the verb, in either direction.
- **Annotations.** A write tool carries `readOnlyHint = false`, `destructiveHint = Destructive` (default `false`),
  `idempotentHint = Idempotent` (default `false`) and `openWorldHint = false`. Declare `Destructive = true` for an
  endpoint that deletes or irreversibly overwrites data. Its description says it changes data and should be called
  only when the user asked for the change.
- **Confirmation is the client's.** `readOnlyHint = false` is what makes MCP clients (Claude Desktop, Claude Code) ask
  the user before running the tool. Beacon does not add its own confirmation step: MCP elicitation is not used for
  write tools, so a client that auto-approves tools runs them without asking.
- **User callers only.** A write tool is listed to, and runs for, a mapped Entra **user** caller only. A **system**
  caller (an app-only token or service user) never sees write tools (not in the tool list, not in `search_api`) 
  and a call to one is refused with `Forbidden: api_<name> changes data in the host application, and write tools run
  only for a signed-in user; a system caller cannot call them.`, whatever `HostClaims` it holds. A system identity
  acts with standing permissions and has no person who asked for the change and no client to confirm it. API-key
  callers cannot call any endpoint tool.
- **The endpoint is the authority.** Beacon checks the caller, the project and the endpoint's authorization policy,
  exactly as for reads, and adds nothing else. The endpoint must do its own input validation, business rules and
  record-level authorization ("only your own leave") against the host principal your `IMcpHostPrincipalFactory`
  builds: treat a tool call like any other request from that user. Antiforgery passes the same way as for reads
  (see [Antiforgery](#antiforgery)), so it is not a defence here.
- **Audit.** Write calls are audited under the tool name `api_<name>:write` (see [Audit and privacy](#audit-and-privacy)).

`IBeaconToolMetadata.Destructive` and `Idempotent` are default interface members: a host's own implementation of the
interface keeps compiling and behaves as `false` for both.

## Who the call runs as

Endpoint tools need a **mapped MCP caller**, an Entra user or system identity from
[Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/), configured for the tools' project and allowed to read it by its
[project access](https://moberghr.github.io/beacon/security/authorization/#project-access): a user caller reaches a restricted project only while its
Beacon user is a member (see [What a caller reaches](https://moberghr.github.io/beacon/mcp/entra-callers/#what-a-caller-reaches)). API-key
callers and callers outside the project neither see nor can call them (fail closed); neither can anyone while the
project does not exist.

`IMcpHostPrincipalFactory` builds the host principal from the caller:

- **System caller** → identity `BeaconMcpSystem` with the caller's configured `HostClaims` plus its name.
- **User caller** → identity `BeaconMcpUser` with name, email, preferred username and object id copied from the
  MCP principal, plus any `HostClaims` the caller mapper assigned (none with the configured mapper). The default
  gives a user **no host permissions**, and deliberately does **not** copy Entra `roles` or `groups`: a host role
  that happens to share a name (`Admin`) would otherwise be granted implicitly. Map users to host permissions in
  your own factory. The name, email and preferred username are **display data**: Entra lets a tenant change them,
  and a guest's email comes from its home tenant, so never map a user to host permissions by one of them. Use the
  object id (`oid`) together with the tenant id (`tid`).
- `null` → the call is refused.

A host whose permissions live in its own user store registers its own factory. The example below maps the Entra user
to the host's admin user and its `AdminPermission` set. The host's permission claim is `HostClaimTypes.Permission`
(`http://schemas.example.com/claims/permission`) and its value is the permission's **integer** value
(`AdminPermission.ViewLoans.ToValueString()` → `"1007"`), which is what the host's `Identity.HasPermission` compares
against. The admin lookup is a **placeholder**: use whatever service the host already has for loading an active
admin and their permissions (it stores the Entra tenant id and object id on the admin record):

```csharp
builder.Services.AddSingleton<IMcpHostPrincipalFactory, HostPrincipalFactory>();

internal sealed class HostPrincipalFactory(IServiceScopeFactory scopes) : IMcpHostPrincipalFactory
{
    public async Task<ClaimsPrincipal?> CreateAsync(McpCaller caller, ClaimsPrincipal mcpPrincipal, CancellationToken ct)
    {
        if (caller.Kind == McpCallerKind.System)
        {
            // HostClaims come from Beacon:Mcp:Callers:Systems[*]:HostClaims: keep the set minimal.
            return new ClaimsPrincipal(new ClaimsIdentity(caller.HostClaims, "BeaconMcpSystem"));
        }

        // Key the lookup on the immutable object id (and the tenant id), never on email, UPN or preferred username.
        var objectId = mcpPrincipal.FindFirst("oid")?.Value;
        var tenantId = mcpPrincipal.FindFirst("tid")?.Value;
        await using var scope = scopes.CreateAsyncScope();
        // PLACEHOLDER: your own lookup of an active admin user and their AdminPermission set.
        var admins = scope.ServiceProvider.GetRequiredService<IYourAdminLookup>();
        var admin = objectId == null || tenantId == null
            ? null
            : await admins.FindActiveAdminByEntraIdAsync(tenantId, objectId, ct);
        if (admin == null)
        {
            return null;                               // not an admin: refused
        }

        var claims = admin.Permissions                 // IEnumerable<AdminPermission>
            .Select(x => new Claim(HostClaimTypes.Permission, x.ToValueString()))
            .Append(new Claim(ClaimTypes.Name, admin.UserName));

        return new ClaimsPrincipal(new ClaimsIdentity(claims, "BeaconMcpUser"));
    }
}
```

System identity host claims, in configuration (the value is the integer `AdminPermission` value; `1007` is
`ViewLoans`):

```json
"Beacon": { "Mcp": { "Callers": { "Systems": [ {
  "Name": "routine-runner",
  "ClientId": "<routine-runner-app-client-id>",
  "ProjectIds": [ 3 ],
  "HostClaims": [ { "Type": "http://schemas.example.com/claims/permission", "Value": "1007" } ]
} ] } } }
```

## How a call is dispatched

1. The caller, project and arguments are checked; the host principal is built.
   The call then takes a slot: at most `MaxConcurrentCallsPerCaller` calls per MCP caller and `MaxConcurrentCalls`
   in total are in flight. A call past either limit waits up to `QueueWait` (5 s) for a slot, then is refused ("Too many host endpoint
   calls are in progress; retry shortly.") and audited. The slot is held until the endpoint really ends, so an
   endpoint that ignores its `RequestAborted` token keeps it after a timeout.
2. A **new DI scope** and a new `HttpContext` are created. The request carries the method, the route filled from the
   endpoint's route pattern, the query string, the declared headers only, and a JSON or
   `application/x-www-form-urlencoded` body. **Nothing of the MCP request is forwarded**: no `Authorization` header,
   no cookies, no connection info. It also sends an `Accept` header preferring JSON (`application/json, text/plain;q=0.9, */*;q=0.1`) and `X-Requested-With: XMLHttpRequest`.
3. The endpoint and route values are set as routing would set them, and checked as routing would check them first:
   a value that fails the route's constraints (`{id:int}`, `{slug:regex(...)}`, a custom `IRouteConstraint`,
   evaluated without an `HttpContext`), a `/` in a single-segment value, or a `.` or `..` segment in any route value
   is refused as invalid arguments before anything runs.
4. **Authorization** runs before the endpoint, exactly as the authorization middleware would, against the host
   principal: the endpoint's `[Authorize]` / policy / requirement metadata combined through the host's
   `IAuthorizationPolicyProvider`, evaluated with `IPolicyEvaluator`. `[AllowAnonymous]` is honoured. No
   authentication handler runs, which is why a policy naming an untrusted scheme is refused at startup (see
   [Authentication schemes](#authentication-schemes)). An endpoint
   without authorization metadata falls back to the host's `FallbackPolicy`; with no fallback policy either it is
   refused at startup (and at dispatch). A denial is the tool error `Forbidden.` with no detail. An endpoint marked
   `[AllowAnonymous]` is exposable and runs without an authorization check: the host itself declared it public, so
   exposing it to an authorized MCP caller grants nothing new.
5. The registered `IHostEndpointDispatchMiddleware` chain runs, in registration order, around the endpoint (see below).
6. The endpoint's `RequestDelegate` runs; for MVC that is the whole action pipeline: model binding, validation,
   authorization/action/exception filters, the result.
7. The response is captured into a buffer capped at `MaxResponseBytes`; the request is aborted after
   `RequestTimeout` or when the MCP call is cancelled. Aborting signals `RequestAborted`; an endpoint that ignores it
   keeps running, with its DI scope and its call slot, until it ends, so slow endpoints that do not flow their
   cancellation token are bounded by the concurrency limits, not by the timeout.

Host code that reads `IHttpContextAccessor` sees the dispatched request. The dispatch runs in its own execution
context, so the MCP request's context (and the audit identity) is untouched.

### Host middleware and ambient state: dispatch middleware

Host **middleware does not run** on dispatch: only the endpoint's own pipeline. When the endpoint depends on state a
middleware sets, register an `IHostEndpointDispatchMiddleware` (namespace `Beacon.Core.HostEndpoints`, shipped in
`Moberg.Beacon.MCP`):

```csharp
Task InvokeAsync(HttpContext syntheticContext, McpCaller caller, Func<Task> next, CancellationToken cancellationToken);
```

Registered middlewares are chained in registration order, with the endpoint's request delegate as the innermost
`next`. The chain starts after `User`, `RequestServices`, the endpoint and route values are set and after Beacon's
authorization check. As in ASP.NET Core middleware, ambient state set before `await next()`
(`Thread.CurrentPrincipal`, an `AsyncLocal`) is visible to the endpoint and its filters; async work before it is
fine. A middleware that does not call `next` refuses the call; one that throws fails it. Both return a generic tool
error and are audited. The dispatch runs in its own execution context, so the MCP request's own ambient principal is
never changed.

Take a host whose `[Permission(AdminPermission.X)]` is an `IAsyncAuthorizationFilter` that checks
`Identity.Current.HasPermission(...)`, where `Identity.Current` reads `Thread.CurrentPrincipal`, which the host's own
middleware sets with `Identity.Set(identity)`: it takes a `ClaimsIdentity`, not a principal. Without a dispatch middleware every tool call would be denied. The
factory above maps the Entra user to admin permissions; the middleware makes that principal ambient:

```csharp
builder.Services.AddSingleton<IHostEndpointDispatchMiddleware, HostIdentityMiddleware>();

internal sealed class HostIdentityMiddleware : IHostEndpointDispatchMiddleware
{
    public async Task InvokeAsync(HttpContext ctx, McpCaller caller, Func<Task> next, CancellationToken cancellationToken)
    {
        // Identity.Set takes a ClaimsIdentity: the identity of the principal IMcpHostPrincipalFactory built.
        Identity.Set((ClaimsIdentity)ctx.User.Identity!);
        await next();
    }
}
```

### Antiforgery

For a call with a body (a write tool, or a read-only tool over POST), Beacon **mints a real antiforgery token pair** for the host principal with the host's own `IAntiforgery` and puts it
on the dispatched request (the cookie token as a cookie, the request token in the configured header, or the form
field when the header is disabled). The host's own validation (`[ValidateAntiForgeryToken]`, a global
`AutoValidateAntiforgeryTokenAttribute`, minimal API form endpoints) therefore runs and passes. For minimal API
endpoints that require validation, Beacon validates the request the way `UseAntiforgery()` does and records the
result. Nothing is exempted and real browser requests are validated exactly as before: the tokens are minted
server-side and never leave the process (CSRF cannot apply to a request that never came from a browser). A host
with a custom anti-CSRF scheme (a bespoke header check, an `IAntiforgeryAdditionalDataProvider` that inspects the
request) must make sure it accepts these requests.

### Responses

- 2xx JSON → the JSON as text plus `structuredContent`. MCP structured content must be an object, so a top-level
  JSON array is wrapped as `{ "items": [...] }` (the text content is the unwrapped JSON).
- 2xx text → text. An empty body → a "no content" line.
- Any other content type → a tool error.
- Non-2xx → a tool error with the status and a short body excerpt.
- An exception in the endpoint → your `IExceptionHandler`s run, in registration order, as `UseExceptionHandler()` runs
  them for a browser request (host middleware does not run for a dispatched call). The response the first one writes —
  the 403, 404 or 400 problem your application maps a deliberate refusal to — is the tool error, its body passed to the
  agent as the handler wrote it. A handler that takes the exception without setting a status answers 500. An exception no
  handler takes is a generic tool error (the message is not returned or logged). Set `UseHostExceptionHandlers = false` to
  always answer the generic error.
- Over `MaxResponseBytes` → a tool error; narrow the request.

## Audit and privacy

Every call (success, forbidden, refused, invalid arguments, timeout, error) writes an MCP audit row with the tool
name `api_<name>` (or `search_api`), the project, the duration, the caller kind and hash, and the outcome. A call to a
**write tool** is recorded as `api_<name>:write` (whether it was called by name or through `call_api`, and also when
it was refused) so writes can be filtered in the audit log (`tool=api_book_leave:write`) without a schema change.
Metrics still bucket it under `api_*`. The
arguments are stored only as the project's content-retention settings allow: under the
[retention lock](https://moberghr.github.io/beacon/mcp/server/) they are reduced to their size. Argument values and response bodies are
never written to logs.

`get_context format=agents_md` lists the project's endpoint tools in the brief (name and first description line):
the ones the caller's tool list offers, as the overview does. Write tools are marked *(writes)*.

## Limits

- Write tools are off unless `AllowWriteTools` is on, run for user callers only, and rely on the client's tool
  approval for confirmation (no Beacon-side approval or elicitation step).
- Host **middleware does not run**: only the endpoint's own pipeline. Recreate what the endpoint needs from it (an
  ambient principal, a tenant, a culture, an admin user loaded into `HttpContext.Items`) in an
  `IHostEndpointDispatchMiddleware`.
  A global MVC `AuthorizeFilter` whose policy names authentication schemes re-authenticates by scheme, finds no
  cookie and denies; express such rules as endpoint metadata (`[Authorize]` / policy attributes) instead.
- Endpoint metadata that names an authentication scheme fails startup unless the scheme is in
  `TrustedAuthenticationSchemes`.
- File uploads and downloads are not supported; collections of objects in form models are not flattened.
- The request host is `localhost` and there is no remote IP address; do not rely on either for authorization.
- Parameter names from custom binders are not known, so their fields are free-form.


---

# Host DbContext

> Expose the host application's own EF Core DbContext as a read-only Beacon data source: default-deny table allow-list, column exclusion and masking, no stored connection string.

Source: https://moberghr.github.io/beacon/mcp/host-dbcontext/

When Beacon is embedded in an existing .NET application, that application's own database can be exposed to
Beacon (and through it to MCP agents) straight from its EF Core model. No connection string is stored, the
metadata comes from the model, and only the tables you allow-list can be read.

## Quick start

```csharp
var beacon = builder.Services.AddBeaconServices(builder.Configuration, o => { /* as today */ });

beacon.ExposeDbContext<ShopDbContext>(o => o
    .ReadOnlyConnection("BeaconReadOnly")            // ConnectionStrings:BeaconReadOnly: a read-only SQL login
    .AllowTables("Loan", "LoanInstallment", "Customer")
    .AllowEntity<Merchant>()
    .ExcludeColumns(x => x.Table == "Customer" && x.Name == "KycNotes")
    .MaskColumns(x => x.Name is "SSN" or "AccountNumber"));

beacon.AddSqlServerConnector();
beacon.UseSqlServer();                                // ends the builder chain: call ExposeDbContext first

var app = builder.Build();
// ... apply migrations ...
await app.Services.SyncBeaconHostDataSourcesAsync();  // creates / refreshes the data source
```

`SyncBeaconHostDataSourcesAsync` is a plain startup call (not a hosted service). It is idempotent: it creates the
project (`ProjectName`, default = the data-source name), the data source and the project link on first run, and
on later runs rewrites metadata only when the hash of the exposed model slice changed.

When the host also ships documentation with [`ExposeDocs`](https://moberghr.github.io/beacon/mcp/host-docs/), call
`SyncBeaconHostAsync()` instead: it runs this data-source sync and then imports the documents, as the single
startup entrypoint. `SyncBeaconHostDataSourcesAsync` keeps working on its own.

## Options

| Option | Meaning |
|---|---|
| `Name` | Data-source name. Default: the context type name. Also the stable key (`efcore:{Name}`). |
| `ProjectName` | Project to create or attach to. Default: `Name`. |
| `Description` | Description used when the project is created. |
| `ReadOnlyConnectionStringName` / `ReadOnlyConnection(name)` | **Required.** Name of a `ConnectionStrings` entry. Validated at startup; resolved from configuration at execution time; never persisted. |
| `Engine` | Normally inferred from the EF provider. Only SQL Server and PostgreSQL are supported. |
| `AllowTables("Table", "schema.Table")` | Allow-list by store name (case-insensitive). Unknown names fail startup. |
| `AllowEntity<T>()` | Allow-list the table or view an entity maps to. |
| `AllowAllTables()` | Expose everything. Deliberately explicit; prefer an allow-list. Logged as a warning at every sync. |
| `ExcludeColumns(Func<HostColumn, bool>)` | Invisible in metadata **and** rejected at execution. |
| `MaskColumns(Func<HostColumn, bool>)` | Visible in metadata (flagged `[PII: values are masked in query results]`); values masked in every result. Exclusion wins. |
| `IncludeSecretLikeColumn("Table.Column")` | Lifts the secret-like hard-exclude for one column. Logged as a warning at every sync. |
| `AllowFunctions("fn", …)` | Allows extra functions (unqualified name, case-insensitive) on top of the built-in allow-list. Only name functions that read nothing but their arguments. Catalog / settings / file / remote functions (`pg_*`, `xp_*`, `sp_*`, `fn_*`, `OPENROWSET`, `query_to_xml`, `row_to_json` …) are refused at startup. |

`HostColumn` gives `Schema`, `Table`, `Name` (column), `ClrType`, `EntityType` and `PropertyName`.

### Secret-like hard-exclude

Regardless of your predicates, a column is excluded when:

- its column or property name **contains** `password`, `passwd`, `passphrase`, `passcode`, `secret`, `token`, `apikey`,
  `api_key`, `privatekey`, `private_key`, `credential`, `connectionstring`, `securitystamp` or `recoverycode` (also with
  an underscore: `connection_string`, `security_stamp`, `recovery_code`);
- it has a **whole word** (camelCase / snake_case) `pin`, `pwd`, `salt`, `hash`, `otp`, `totp`, `hotp`, `mfa`,
  `refresh`, `pfx`, `nonce`, `seed`, `cvv` or `cvc`, so `PasswordHash`, `PinCode`, `OtpSeed`, `TotpKey` and
  `RefreshToken` are excluded while `Shipping` is not;
- it has a `key` word beside a word naming key material (`api`, `access`, `secret`, `private`, `signing`, `encryption`,
  `authenticator`, `certificate`, `cert`, `master`, `license`, `hmac`, `aes`, `rsa`, `pgp`, `gpg`, `ssh`):
  `EncryptionKey` and `SigningKey` are excluded, `ProductKey` is not;
- its **table** holds credentials: `AspNetUserTokens`, `DataProtectionKeys`, or any table whose name ends in the word
  `Keys` or `Tokens` (`ApiKeys`, `signing_keys`, `RefreshTokens`) — every column of it;
- its entity is ASP.NET Core Identity's `IdentityUserToken<TKey>` or Data Protection's `DataProtectionKey` (or derives
  from one), whatever its table is called — every column of it.

Business words that only sometimes name a secret (`SigningDate`, `CertificateNumber`, `RecoveredAmount`, `TimeStamp`,
`ConcurrencyStamp`) are not excluded on their own. Only `IncludeSecretLikeColumn` lifts the exclusion. Every sync logs
the columns it excluded (`schema.Table.Column`). The rule is a best-effort name heuristic: also deny the secret columns
to the read-only login.

## Metadata

Read from the EF design-time model without opening a connection: tables and views (`ToTable` / `ToView`),
schemas, column names and store types, nullability, max length, primary keys, indexes, and **foreign keys**
(when the principal table is also exposed; they feed the schema-relationship graph and join suggestions).
Descriptions come from `HasComment`, falling back to XML documentation (`<summary>`) when the entity assembly
ships its `.xml` file (`<GenerateDocumentationFile>true</GenerateDocumentationFile>`). Owned types and table
splitting are merged into their table; entities mapped to SQL queries or functions are skipped; keyless views are
included when allow-listed.

A live catalog scan and sample-value collection are never run for a host data source; "Refresh metadata" in the
UI is refused; restart the host (or call the sync again) instead.

## Security model

- **No stored secret (§1.1).** `EncryptedConnectionData` holds only an encrypted reference,
  `{"hostConnectionStringName":"BeaconReadOnly"}`. The real string is read from `IConfiguration` whenever a
  connection is opened (`IDataSourceConnectionResolver`, the single decrypt-and-resolve point).
- **Host-owned.** The data source is marked `IsReadOnly`, cannot be edited or deleted through the REST API/UI
  (the request is refused), and cannot be used by migration jobs. Descriptions, glossary, golden queries and the
  knowledge graph work as for any other data source.
- **Policy enforced at execution.** Every SQL statement against a host data source is checked against the
  exposed slice, in the execution gate (MCP `query`, `dry_run`, cross-source, eval; a violation blocks regardless
  of `BlockOnSchemaFailure`) **and** again in the database provider (every path: `ask`, query previews, value
  grounding, documentation sampling), in saved-query steps and in data-quality rules. The check parses the SQL in
  the data source's own dialect and resolves every name the way the database does:
  - only a single `SELECT`; every table must be allow-listed **and schema-qualified** (`dbo.Customer`,
    `public."Customer"`), an unqualified name is rejected with `Qualify the table: dbo.Customer`, because the
    connection's real default schema / `search_path` may differ from the model's. Three-part / linked-server names,
    table-valued functions, `OPENROWSET` / `OPENJSON`, `SELECT INTO`, `FOR JSON/XML`, locking clauses, `NATURAL`
    joins and temporal `FOR SYSTEM_TIME` are rejected;
  - identifiers compare as the engine compares them (PostgreSQL: unquoted names fold to lower case, quoted names are
    exact; SQL Server: case-insensitive), for tables, CTE names, aliases and columns alike, so a CTE never hides a
    real table (`WITH "AuditLog" AS (…) SELECT … FROM auditlog` is rejected);
  - on PostgreSQL, every identifier is read as the server names it: an identifier longer than 63 bytes in UTF-8 is
    shortened to 63 bytes on a character boundary, as PostgreSQL silently does (quoted names too, output aliases
    included), and compared and masked by that name. Only what the server would read differently than written is
    rejected: two different identifiers of the statement that are the same in their first 63 bytes (PostgreSQL would
    read them as one; use names that differ sooner, or, when the host's own names collide, expose a view or rename the
    column on the host), and an unquoted identifier with a non-ASCII letter that changes case, such as `Šifra` or the
    Kelvin sign (PostgreSQL folds only `A`–`Z`; write it in lower case or quote it, `"Šifra"`);
  - names resolve with per-`SELECT` scopes: an unqualified column must be an exposed column of a relation in that
    `SELECT`'s own `FROM`; a correlated (outer) reference must be qualified with the outer alias; output aliases are
    visible only to the same `SELECT`'s `ORDER BY` (and PostgreSQL's `GROUP BY`) and to outer queries through a
    derived table's / CTE's columns. A bare relation name is never a column, so whole-row references
    (`SELECT c FROM … c`, `row_to_json(c)`, `(c).col`) are rejected; unknown and excluded columns are rejected
    anywhere; a table alias may be used only once per query;
  - on PostgreSQL a bare `GROUP BY` name is read the way the server reads it: as a column of the `SELECT`'s own
    relations first, and as an output alias only when none of them has a column of that name. A table can have
    columns the policy excludes or the model does not map, so an output alias in `GROUP BY` is accepted only when the
    `SELECT` reads derived tables or CTEs alone; over a table, group by the expression itself or by its position
    (`GROUP BY 1`);
  - on PostgreSQL, catalog types (`regclass`, `regrole`, `regproc`, `regtype` … and their arrays, `regclass[]` or
    `_regclass`, in any case, quoted or schema-qualified) are rejected as casts (`CAST(… AS regclass)`,
    `'…'::regclass`) and as typed literals (`regclass '…'`), and a string where a name belongs (`SELECT col 'x'`,
    which PostgreSQL reads as a typed literal) is rejected — quote names with double quotes;
  - `SELECT *` / `t.*` is rejected with a list of the exposed columns to use instead;
  - functions must be on a per-dialect **allow-list** of built-in string, date/time, math, conditional, conversion,
    aggregate, window and (on column values) JSON functions, plus any `AllowFunctions` entries; everything else,
    user-defined functions included, is rejected. Bind parameters are `@p0`, `@p1` … on both engines (the bare
    PostgreSQL `@` operator is rejected; use `abs()`), and a statement never runs with a parameter unbound;
  - string-building functions take only bounded literals, so one call cannot build a huge value:
    - the count of `repeat` (`REPLICATE` on SQL Server) and the width of `lpad` / `rpad` must be a number literal of
      at most 10,000, optionally quoted (`'5'`) or with one cast straight over it to an integer or numeric type
      (`'5'::int`, `CAST(5 AS bigint)`); a column, parameter, expression, a cast to another type or a cast of a cast is
      rejected, and a negative number passes, since it builds nothing;
    - the string `repeat` / `REPLICATE` repeats must be a plain string literal of at most 100 characters, so a column
      or a nested call cannot multiply the length (`lpad` / `rpad` cut their string to the width, so theirs is free);
    - PostgreSQL's `format()` needs a string-literal format string whose field widths add up to at most 10,000 and
      that uses sequential specifiers only (`%s`, so each argument is used once; a positional `%1$s` and a `*` width,
      taken from an argument, are rejected);
    - a call that can lengthen its input many times — `repeat` / `REPLICATE`, `replace` / `REPLACE`, `regexp_replace`
      and a `format()` that quotes (`%I` / `%L`) — may not appear anywhere inside an argument of another such call, so
      lengths add up instead of multiplying (`replace(replace(…))` is rejected; one `replace(col, ',', ';')`, and
      `lpad`, `left`, `substring` … around or inside it, pass);
    - SQL Server's `FORMAT` needs a string-literal format string with no number above 999 (a precision such as
      `N999999999` is rejected).

    Other costly work (regular expressions, large joins and aggregates, a value built up step by step through CTEs) is
    bounded by the statement timeout, the row cap and the database's own limit on one value (1 GB on PostgreSQL), not
    by the check — give the host login a server-side statement timeout too;
  - a masked column may appear **only** as a direct projection (optionally aliased, through CTEs, derived tables
    and `UNION ALL` the mask follows the value to its output key), inside `COUNT(col)`, or in `col IS NULL` /
    `col IS NOT NULL`. Any other use (`WHERE`, `JOIN … ON`/`USING`, `GROUP BY`, `ORDER BY` (by name, alias or
    ordinal), `DISTINCT`, `UNION`/`INTERSECT`/`EXCEPT`, window specifications, `COUNT(DISTINCT …)`, any other function,
    `CASE`, casts, or a subquery used as a value) is rejected, so masked values cannot be probed.
  Host-masked values are masked **fully** in every result (`***`; `NULL` stays `NULL`). The result is masked by
  **output position**, not by column name: the check knows which columns of the statement carry a masked value, and
  the provider masks those columns of the result whatever the database calls them: under any alias, a repeated
  name or another case. Names serve only as a consistency check that fails closed: a result column the database names
  differently than the check derived is masked too (logged as a warning with the data source id and counts), and a
  result whose column count differs from the statement's is discarded (`The result's columns did not line up with
  the columns the host data source policy checked, so the result was discarded.`). Every path that runs SQL on a host
  data source reads it through the provider: previews (paged ones included), subscriptions and their attachments,
  MCP tools, saved-query tools and the rows the in-memory join reads, so all of them return the masked values.
- **Errors stay generic.** When a statement fails on the host database the caller gets
  `Query failed on the host database.` and the log records only the exception type and the driver's error code;
  server messages can quote column values.
- **Read-only login.** SQL Server has no read-only transaction mode, so `SupportsDatabaseReadOnlyEnforcement`
  stays `false`; the parser gates plus the dedicated read-only login are the guarantee. On PostgreSQL the
  `READ ONLY` transaction backstop applies as usual.
- **Login check at startup.** After every host data source is synced, Beacon asks each host database whether its
  read-only login can write, with fixed catalog statements outside the SQL gate (the exposed tables' names are
  parameters):
  - **PostgreSQL:** whether the login is a superuser (`rolsuper`) or holds `INSERT`, `UPDATE` or `DELETE` on an exposed
    table (`has_table_privilege`), including column-level `INSERT` or `UPDATE` (`has_any_column_privilege`).
  - **SQL Server and Azure Synapse:** whether the login is in `db_owner` or `db_datawriter`, holds a database-wide
    `INSERT`, `UPDATE` or `DELETE` permission, or holds one of them on an exposed table
    (`HAS_PERMS_BY_NAME(..., 'OBJECT', ...)`); the database-wide probes name the current database (`DB_NAME()`).

  Only those privileges are asked about (not `TRUNCATE`, DDL or other schemas' tables). A login that can write gets one
  Warning per data source naming it by id, never the connection string or the login. A check that cannot run, an answer
  the database does not give, or a source none of whose exposed tables the database finds logs that the login could not
  be verified: nothing checked is never reported as read-only. A connection the transport policy refuses is logged as
  not checked, and a disabled source is not checked at all (it makes no connection). The checks run in parallel, each
  bounded at 10 seconds, so startup waits about 10 seconds at most; they never fail or block startup.

### Behaviour to know

- Beacon knows the columns of the EF model. A statement that references a column the model does not have is
  rejected. Grant the read-only login `SELECT` only on the allow-listed tables (and `DENY` secret columns) so the
  database applies the same boundary.
- A function added with `AllowFunctions` runs with the read-only login's rights. Grant the login `EXECUTE` only on
  the functions you allow.
- A masked column still answers `IS NULL` and `COUNT(col)`. Exclude the column instead when its presence must stay
  hidden too.
- Identifiers are compared case-insensitively, also on case-sensitive SQL Server collations; a name whose case does
  not match fails on the server.
- These forms are refused with a validation message: unqualified correlated references, the same alias in an inner
  and an outer query, `NATURAL` joins, `DISTINCT` or set operations over masked columns, and on PostgreSQL two
  identifiers that share their first 63 bytes or an unquoted identifier with a non-ASCII letter that changes case.
- The 63-byte rule assumes a stock PostgreSQL build (`NAMEDATALEN` 64) and counts UTF-8 bytes. SQL Server keeps
  names up to 128 characters as written, so no length rule applies there.

:::caution[Upgrading]
Host column masking now works by output position. On PostgreSQL host data sources, a statement with two identifiers
that are the same in their first 63 bytes, or with an unquoted identifier that has a non-ASCII letter that changes
case, is now refused with a validation message: use names that differ sooner, write the name in lower case or quote
it. A single identifier longer than 63 bytes is accepted and read as PostgreSQL shortens it. Check saved queries,
subscriptions and saved-query tools on PostgreSQL host data sources for such aliases. A result column the database
names differently than the check derived is now masked (the MCP `query` tool notes how many), and a host data source
result whose columns do not match the statement now fails instead of being returned.
:::

## Example: restricted tables and masked columns

```csharp
beacon.ExposeDbContext<ShopDbContext>(o =>
{
    o.Name = "Shop";
    o.ProjectName = "Shop Back Office";
    o.ReadOnlyConnection("BeaconReadOnly")
        .AllowTables("Loan", "LoanInstallment", "Merchant", "Customer")
        .ExcludeColumns(x => x.Table == "Customer" && x.Name.StartsWith("Kyc"))
        .MaskColumns(x => x.Name is "SSN" or "AccountNumber" or "Email" or "Phone");
});
```

`User.Secret`, `ProviderIntegrationKey.SecretKey` and `Customer.PartnerAuthorizationToken` are excluded by the
hard-exclude even if their tables are allow-listed; the national-id (`SSN`) columns are visible to the agent as
PII-flagged columns and masked in every result.


---

# Host Docs

> Ship the host application's markdown documentation with it and import it into a Beacon project. The documents are searchable by MCP agents with or without embeddings, readable through get_documentation, and summarised in an AGENTS.md project brief.

Source: https://moberghr.github.io/beacon/mcp/host-docs/

A host application often keeps its own documentation next to the code: a `docs/wiki/` folder of generated markdown
pages, runbooks, conventions. `ExposeDocs` imports those documents into a Beacon project at startup
so MCP agents can `search` them and read them with `get_documentation`, alongside the host's data
([Host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/)). It works with `UseAI = false`: no LLM and no embedder is needed.

## Quick start

```csharp
var beacon = builder.Services.AddBeaconServices(builder.Configuration, o => { /* your options */ });

beacon.ExposeDbContext<HostContext>(o => o
    .ReadOnlyConnection("BeaconReadOnly")
    .AllowTables("Loan", "LoanInstallment", "Customer"));

beacon.ExposeDocs(o =>
{
    o.ProjectName = "HostContext";             // the project ExposeDbContext uses (its Name/ProjectName)
    o.FromDirectory("docs/wiki")               // relative to ContentRootPath; default glob **/*.md
     .Exclude("drafts/**");
});

beacon.UseSqlServer();                         // ends the builder chain: call ExposeDocs first

var app = builder.Build();
// ... apply migrations ...
await app.Services.SyncBeaconHostAsync();      // syncs ExposeDbContext data sources, then ExposeDocs documents
```

`SyncBeaconHostAsync` is the single startup entrypoint for everything a host declares. It runs the
`ExposeDbContext` sync first (exactly what `SyncBeaconHostDataSourcesAsync` does; that call still works on its
own) and then imports the documents. It is a plain startup call, not a hosted service.

Both calls are **safe to run from several replicas at once**: the whole sync holds one lock in Beacon's own database
(PostgreSQL `pg_try_advisory_lock` on a fixed key, SQL Server `sp_getapplock` with a session owner), so replicas
starting together run it one after another instead of racing each other's inserts. A replica waits up to 5 minutes
for the lock. A unique-key race that still happens (a row created outside the lock) re-runs the idempotent sync once
instead of failing startup. Other providers run the sync unlocked and log a warning. A host that coordinates startup
another way can register its own `IHostSyncLock`.

### The host project

`ExposeDbContext`, `ExposeDocs` and `AddHostEndpointTools` all resolve their `ProjectName` through one shared
resolver, so they always land on the same project. The host project is identified by a **host key**
(`Project.HostManagedKey = host:{ProjectName, trimmed and lower-cased}`), never by its display name, which is neither
unique nor protected:

- On first sync the host project is created with the key. If a project that is **not** host-managed already uses the
  name (for example one a Beacon user created by hand), it is left untouched and the host project is created as
  `"{ProjectName} (host)"` (then `"(host 2)"`, …), with a warning in the log. A user project with the same name never
  receives the host database, documents or endpoint tools.
- Renaming the host project in the UI is fine: it is still found by its key.
- An archived host project is restored, because the host still declares it.
- Changing `ProjectName` creates (or claims) the project for the new name, and the host data source is **unlinked
  from every other project**, including the project of the old name, so it loses access to the host database.

## Shipping the documents

The docs are not part of the build output by default, so the host has to ship them. Either copy them next to the
app:

```xml
<!-- YourHost.csproj -->
<ItemGroup>
  <Content Include="..\docs\wiki\**\*.md" CopyToOutputDirectory="PreserveNewest" LinkBase="docs/wiki" />
</ItemGroup>
```

```csharp
o.FromDirectory("docs/wiki");
```

`FromDirectory` resolves a relative path against `IHostEnvironment.ContentRootPath`. Make sure that is the
output directory in your deployment (it is when the app is started from its own folder), or pass an absolute
path such as `Path.Combine(AppContext.BaseDirectory, "docs/wiki")`.

Or embed them in the assembly:

```xml
<ItemGroup>
  <EmbeddedResource Include="..\docs\wiki\**\*.md" LogicalName="docs/wiki/%(RecursiveDir)%(Filename)%(Extension)" />
</ItemGroup>
```

```csharp
o.FromEmbeddedResources(typeof(Program).Assembly, "docs/wiki/");
```

The part of the resource name after the prefix is the document path (backslashes become `/`). Without a
`LogicalName`, MSBuild mangles folder names (`40-web` becomes `_40_web`) and joins them with dots, so set one.

## Options

| Option | Meaning |
|---|---|
| `ProjectName` | **Required.** The host project the documents attach to, found by its host key (see [The host project](#the-host-project)); created when missing. |
| `FromDirectory(path, params globs)` | Files under `path` (relative to the content root, or absolute) matching the globs. Default glob `**/*.md`. |
| `FromEmbeddedResources(assembly, prefix)` | Manifest resources whose name starts with `prefix`. |
| `Exclude(params globs)` | Skips documents whose relative path matches (applies to every source of this call). |
| `MaxDocumentBytes` | Per-document cap, default 512 KB. Larger files are skipped with a warning that names only the path. |

`ExposeDocs` may be called more than once, for several folders or for several projects. Only `.md`,
`.markdown` and `.txt` files are ingested, whatever the glob matches. Globs must be relative and may not contain
`..`; a file that resolves outside the configured directory, and any symbolic link inside it (file or folder), is
rejected. A configured directory that does not exist fails startup, so a broken deployment cannot silently archive
every imported document.

## What the sync does

Each document is stored as a `ProjectImportedDocument`, keyed by project, source (`docs:dir:docs/wiki`,
`docs:res:{Assembly}:{prefix}`) and relative path.

- **Title**: the frontmatter `title`, else the first `# ` heading (outside code fences), else the file name.
- **Frontmatter**: a leading `---` YAML block is stripped from the body and kept as JSON (flat keys, quoted
  strings, `[a, b]` and `- item` lists, such as a `sources:` list).
- **Idempotent**: a document whose content hash is unchanged is left alone; nothing is written when nothing
  changed. A changed document is updated in place, a removed file is archived, and a file that comes back is
  un-archived.
- **Case-insensitive paths**: source and path are matched ignoring letter case, as SQL Server's unique index
  compares them. Renaming `Guide.md` to `guide.md` updates the existing row (it takes the new casing). Two files
  whose paths differ only by case (possible on a case-sensitive file system or in embedded resources) cannot both be
  stored: the first is imported and the other is skipped with a warning that names its path.
- **Chunked for search**: new and changed documents are split into sentence-window chunks (the same chunker and
  `DocChunkWindowSentences` / `DocChunkOverlapSentences` settings as generated documentation). Keyword search
  works on them immediately. When the local embedder and semantic retrieval are enabled, the sync then embeds the
  new chunks through the doc-chunk re-index, and the recurring `mcp-docchunk-reindex` job keeps them current. An
  embedding failure is logged and does not fail startup.

The documents belong to one project and are only ever visible to callers authorized for that project.

## On the MCP surface

- **`get_documentation`**: the project-level response (no arguments) ends with an *Imported documents* listing
  (title + path). `document: "<path>"` (or an exact title) returns one document in full; a title shared by several
  documents is an error that names their paths.
- **`search`**: imported documents match on title, path or chunk text (any word of the query; the documents holding
  the most words first) and appear as
  `- **[IMPORTED_DOC]** \`path\` (title)`, followed by ` -- snippet` when there is one, with or without embeddings.
- **`get_context` with `format: "agents_md"`**: a deterministic *project brief* in markdown to drop into an agent
  workspace's `AGENTS.md`: the project description, data sources (host-managed ones marked read-only), up to 30 key
  tables (best-described and most-related first), masked columns, up to 30 glossary terms, the imported document
  titles, and how to use Beacon's tools (`get_query_context` → `dry_run` → `query`, read-only, masked columns stay
  masked; a project without data sources is pointed at `search` and `get_documentation` instead). Built from the
  catalog only, with no LLM call, and audited like every tool call. The brief opens with a notice that catalog text
  is untrusted data, prints the project description as one truncated code-span line, and renders every other
  catalog-derived field (names, identifiers, document paths, glossary text, tool descriptions) as data: on one line,
  with hidden characters shown as `�`, identifiers and paths in code spans no backtick can close, and Markdown
  delimiters escaped. It states the read-only and PII-detection guarantees from the project's effective MCP settings
  (it says so when either is switched off); the masked columns it lists are a host-managed source's host-policy
  masked columns, which stay masked whatever the PII detection setting.

The REST API exposes the same documents: `GET /beacon/api/projects/{id}/imported-documents` (listing) and
`GET /beacon/api/projects/{id}/imported-documents/{documentId}` (one document with its content). Both need Read in
the project under [project access](https://moberghr.github.io/beacon/security/authorization/#project-access), for sessions and API keys alike; a
project the caller cannot read answers `404`.

## Example: data and docs in one project

```csharp
beacon.ExposeDbContext<HostContext>(o =>
{
    o.Name = "BackOffice";
    o.ProjectName = "Back Office";
    o.ReadOnlyConnection("BeaconReadOnly").AllowTables("Loan", "LoanInstallment", "Merchant", "Customer");
});

beacon.ExposeDocs(o =>
{
    o.ProjectName = "Back Office";
    o.FromDirectory("docs/wiki");
});
```

With the `Content Include` item above, the pages land in `docs/wiki/` next to the host's assembly. On each start only
the pages that changed since the last deployment are re-chunked.


---

# Knowledge Base & SQL Grounding

> How Beacon grounds generated SQL: M-Schema with sample values, schema relationships, a business glossary, human-verified golden examples, learned patterns, and semantic retrieval.

Source: https://moberghr.github.io/beacon/mcp/knowledge-base/

Beacon does not pass a bare schema dump to an LLM. Every natural-language question (through the [`ask` tool](https://moberghr.github.io/beacon/mcp/server/#ask), the [MCP Playground](https://moberghr.github.io/beacon/mcp/server/#mcp-playground), or the [`get_query_context` tool](https://moberghr.github.io/beacon/mcp/server/#get_query_context)) is answered against an assembled **grounding context**: the tables that matter, the join paths that are real, the business terms your organisation actually uses, and query examples a human has verified.

This page describes what goes into that context, where each piece comes from, and how you curate it.

## The grounding context

For a given question and data source, Beacon assembles the context in blocks. Sections marked **(authoritative)** are human-verified; everything else is machine-derived and labelled as such.

| Block | Contents | Source |
|---|---|---|
| **Relevant Tables (full schema)** | M-Schema rendering: column name, type, nullability, description, plus **real sample values** per column | Schema metadata + sample-value collection |
| **Other Tables** | Names and primary keys only, so the model knows what else exists without burning the budget | Schema metadata |
| **Join Paths (verified)** | Foreign keys and human-confirmed relationships | [Schema relationships](#schema-relationships) |
| **Join Paths (UNVERIFIED)** | Relationships pending curator verification, explicitly flagged as needing confirmation: an inferred one with its confidence, one a user added as *added, pending curator verification* | Relationship discovery, people who may not curate the source |
| **Coverage** | A note when the table neighbourhood was capped, so the model knows the picture is partial | Context assembly |
| **Business Glossary** | Top-K matching terms with definitions and their column/metric mapping | [Business glossary](#business-glossary) |
| **Verified query examples (authoritative)** | Human-verified question → SQL pairs for this data source | [Golden examples](#golden-examples-and-the-eval-harness) |
| **Learned Patterns (from usage)** | Known corrections, column clarifications, join patterns, common queries, business-term mappings, documentation gaps | [The learning loop](#the-learning-loop) |

Sample values are what make filters land on real data (`'shipped'` vs `'SHIPPED'`). They are row data sent to the LLM provider, so collection is off by default: turn `EnableSampleValueCollection` on for a project (its override on **MCP Settings**, `/beacon/mcp-settings`) or globally. A refresh collects while it is on in the effective settings of every live project that links the data source (each project's override, else the global value), so a project that turns it on collects even while the global value is off. On a data source several live projects link, any of them switching it off stops the collection and keeps the stored values out of every project's context (see [Guardrails of shared data sources](https://moberghr.github.io/beacon/security/authorization/#guardrails-of-shared-data-sources)), and the deployment lock `Beacon:Mcp:ForbidRowDataToLlm` keeps it off everywhere. Documentation generation discovers column values for the same projects only, and logs once per run which data sources it listed no values for because collection is off. PII, secret-like and financial-amount columns are never sampled, and a column with a PII- or credential-shaped value keeps none. See [Sample values](https://moberghr.github.io/beacon/features/data-sources/#sample-values) and [What reaches the LLM provider](https://moberghr.github.io/beacon/mcp/server/#what-reaches-the-llm-provider).

:::tip
Everything on this page is reachable from the outside: `get_query_context` returns the same assembled context an `ask` call would use, so an agent can write its own SQL against the same grounding, validate it with `dry_run`, and execute it with `query`.
:::

## Who curates grounding

What a person approves here ends up in every caller's prompts, so curating it is a narrower right than writing:

| Grounding | Who curates it |
|---|---|
| A project's learned patterns (approve, reject, set back to pending), its documentation (editing a section, regenerating it) and documentation patches to a section | **Curate** in the project: a project admin member of it, or an Admin |
| A data source's shared grounding: verifying or un-verifying a relationship, deleting a verified one, applying or rejecting a documentation patch to a column or table description | Write where the caller acts **and** Curate in **every** live project linking the source, or the DataSourceAdmin permission (with Write where it acts), or an Admin; a source no live project links: a DataSourceAdmin or an Admin |
| Golden examples (promote, edit, deactivate), the [grounding recheck](#rechecking-stored-grounding) and the business glossary | Admins only |

Curate is never granted by an open project, and never to an API key (whatever its scope), an MCP caller (user or system, Read or Execute scope) or the owner of scheduled work, whatever their role. Beacon's own background jobs run without a caller and no project check applies to them, but none of them curates: the learning job stores new patterns as `NeedsEvidence`, auto-approves only through the [replay gate](#from-signal-to-injected-lesson) and proposes documentation patches without applying them. A curation that records who decided (reviewing a learned pattern, promoting or editing a golden case, verifying a relationship) also needs a signed-in Beacon user (`403` otherwise). A host that wants only Admins to curate simply adds no project admins; Beacon [warns at startup](#rechecking-stored-grounding) about live projects that learn from feedback while no enabled user is their project admin. The pattern, patch and relationship lists and the project documentation tell each caller per row whether they may curate it (`canCurate`), decided the same way as the routes (`403` otherwise); the UI shows the actions only then, and says who may otherwise. See [Authorization → Project roles](https://moberghr.github.io/beacon/security/authorization/#project-roles).

**Grounding SQL is checked and stored without comments.** Every piece of SQL that grounds generation (gold SQL that is promoted or edited, corrected SQL from feedback, a learned pattern's example SQL when it is mined, approved or auto-approved) passes the grounding SQL check (`IGroundingSqlCheck`) before it is stored. The SQL is read in its data source's dialect, its comments are removed, and the comment-free SQL must pass the read-only gate and the catalog check a saved query step passes (a single read-only statement, tables and columns Beacon's metadata holds). Nothing runs. The check refuses, with the reason:

- **A comment that cannot be removed safely**: an executable comment (`/*! … */` on MySQL), a comment that is never closed, SQL the dialect's tokenizer cannot read. Comment characters inside a string literal are text, not a comment, and are kept when the dialect's tokenizer reads the literal as the database does: `'a--b'` and `'/* … */'`, `$$…$$` on PostgreSQL, `N'…'` on SQL Server, `'#1'` on MySQL, `'a//b'` on Snowflake, `"a#b"` on BigQuery. Where the engine reads a literal differently from the tokenizer, the SQL is refused, as with `'a--b'` on MySQL, which reads `--` without a following space differently. Comment characters in a quoted identifier (`"a--b"` on PostgreSQL) are refused, and so is any comment marker on a data source whose SQL dialect Beacon does not know.
- **Characters a reviewer cannot see**, anywhere in the SQL, a literal or a comment included: format characters (bidirectional overrides and isolates, zero-width characters, the byte order mark), the tag characters U+E0000–U+E007F and variation selectors.
- **A data source Beacon holds no tables or columns for**: there is nothing to check the SQL against, so it is refused until metadata is extracted ("Extract metadata for this data source first").
- **SQL the read-only gate or the catalog check refuses**: anything but one read-only statement, or a column the data source's metadata does not hold.

A golden case's question holding characters a reviewer cannot see is refused where it is promoted or edited, and a mined example question holding them is not stored.

Prompts render gold and example SQL without comments too, including SQL stored before this check existed, and questions on one line. A golden case whose SQL cannot be shown that way, or whose SQL or question holds characters a reviewer cannot see, is left out whole; a pattern loses only its example; a glossary term loses only its metric expression. Beacon logs which ids it left out, by kind, once per hour per row. The documentation export (behind knowledge answers, `get_documentation` and the documentation chunk index) and the prompt that generates documentation show a pattern's example SQL the same way, or not at all. The golden and pattern lists say which rows a prompt leaves out (`goldSqlWithheld`, `questionWithheld`; `exampleSqlWithheld`, `exampleQuestionWithheld`), and the MCP Learning page flags them and shows hidden characters escaped.

### Rechecking stored grounding

Grounding stored before these checks existed is not rewritten: it keeps grounding prompts, rendered as above, until an Admin decides about it. `GET /beacon/api/eval/grounding/recheck?projectId={id}` (Admin by live roles, like the rest of `/eval`) runs today's checks over one project's grounding and changes nothing: the question and gold SQL of each active golden case, and the example question and example SQL of each approved or auto-approved learned pattern that is not superseded. SQL passes the grounding SQL check in its data source's dialect (each data source and its catalog are read once per recheck); a golden case's question must not be empty, and no question may hold characters a reviewer cannot see. The answer counts what was checked (`goldenCasesChecked`, `patternsChecked`) and lists every row a check refuses (`refused`, golden cases first): its `kind` (`GoldenCase` or `LearnedPattern`), `id`, `dataSourceId`, the stored `question` and `sql`, the reasons (`questionRefusal`, `sqlRefusal`), a golden case's `promotedByUserId` (null when it was promoted before Beacon recorded who promoted it) and a pattern's `patternStatus` and `contentHash`. A project that does not exist answers `404`. The recheck logs counts and ids only.

On the **MCP Learning** page (`/beacon/mcp-learning`) an Admin's **Grounding recheck** tab runs it for a project and lists the refused rows, each with the action the Admin chooses: **Deactivate** a golden case (`PUT /beacon/api/eval/golden/{id}` with `isActive: false`) or set a pattern **Back to pending** (`PUT /beacon/api/mcp/learned-patterns/{id}/status` with `Pending` and the row's `contentHash`). Nothing is deactivated or set back automatically.

**Startup warnings.** After the host starts, Beacon warns (best effort: a database it cannot read is noted, never a failure; counts and project ids only) about grounding that waits for an Admin:

- the active golden cases that record no promoting user (promoted before Beacon recorded who promoted a case), with their projects: they keep grounding SQL generation, so review them on the **Golden examples** tab, run the recheck, and deactivate the ones nobody stands behind;
- with Beacon user management on and learning on (`EnableLearning`, globally and in the project's effective settings), the live projects with no enabled ProjectAdmin member: only an Admin reviews their learned patterns and documentation patches until a member is made their project admin.

## Schema relationships

Join paths are the single biggest source of wrong SQL, so Beacon keeps them as first-class, curatable data rather than inferring them at generation time.

Each registered relationship maps a source column to a target column and carries an **origin** and a **cardinality**:

| Origin | Meaning | Verified by default |
|---|---|---|
| **Foreign key** | Read from the database's declared constraints | Yes |
| **Inferred** | Proposed from column-naming heuristics (`customer_id` → `customers.id`) with a confidence score | No |
| **Manual** | Declared by a person in the UI or via the API | Only when its creator [curates the source](#who-curates-grounding) |

Cardinality is one of `Unknown`, `One to one`, `One to many`, `Many to many`.

Only **verified** relationships are rendered under *Join Paths (verified)*. Unverified relationships appear in a separate block, *pending curator verification*, that tells the model to confirm before relying on them. A wrong join path that looks authoritative is worse than no join path at all.

A verified relationship is a join every project linking the data source is told to trust, so only a curator of the source's shared grounding verifies one or deletes one. A relationship someone else declares (or accepts from discovery) is stored **unverified**, pending verification by a curator (the create answer says so with `isVerified: false`). A verified relationship that someone else relabels or gives another cardinality goes back to unverified (the update answer says so the same way). A curator's own declarations and edits stay verified, and a curator's edit records the curator as the relationship's verifier, now. Unverified relationships stay in the prompt, under the unverified block: an inferred one with its confidence, one a user added as *added, pending curator verification*.

Every change applies to the relationship as it was read. An edit writes the new label and cardinality together with the verification it leaves, in one statement, and only while the relationship still holds the label, cardinality and verification read; a removal is conditional the same way; a verification names the label and cardinality the curator reviewed (`reviewedLabel`, `reviewedCardinality`) and is written only while the relationship holds them. Anything that lands in between (a curator verifying while an Editor's edit or removal is under way, an Editor's edit while a curator verifies) refuses the later request with `409` ("The relationship changed since you loaded it"), so a relationship is never left verified with content its verifier did not see, and a relationship verified meanwhile is never removed on the strength of a check made while it was unverified. Reload the relationships and try again.

### Managing relationships in the UI

Open a data source and go to **Schema relationships** (`/beacon/data-sources/{id}/relationships`):

- **Discover** previews inferred relationships without saving anything. Review each proposal with its confidence score and **Accept** the ones that are right.
- **Add relationship** declares one by hand. It is verified when you curate the data source; otherwise the page says it is pending verification by a curator, and so does an edit that sends a verified relationship back to unverified.
- Existing entries can be verified, un-verified (curators only: for anyone else the action is disabled, with the reason), relabelled or deleted. A verified entry is deleted by curators only; for anyone else the action is disabled, with the reason.

The page opens with a **schema health** panel: table count, relationship count, how many are still unverified, the number of connected groups, the largest connected group, isolated tables, and detected junction tables. Isolated tables and a high group count are the signal that the model has no way to join its way to those tables.

### REST API

The relationship endpoints sit under the standard authenticated `BeaconApi` policy. They manage grounding metadata that every user's questions depend on, so they are not an admin-only surface. Creating and relabelling a relationship need Write in every live project linking the data source, deleting one Manage there (a DataSourceAdmin: where it acts); verifying one, and deleting a verified one, need [Curate](#who-curates-grounding) there (`403` otherwise). The list returns `canCurate` per relationship; the create answer (`id`, `isVerified`) and the update answer (`200`, `isVerified`) say whether the relationship is verified. The verify body is `isVerified` with, to verify, the `reviewedLabel` and `reviewedCardinality` the curator reviewed (`400` without them); an edit, a removal or a verification of a relationship that changed since it was read answers `409`.

| Method | Endpoint |
|---|---|
| `GET` | `/beacon/api/data-sources/{dataSourceId}/relationships` (filter by `origin`, `verifiedOnly`) |
| `POST` | `/beacon/api/data-sources/{dataSourceId}/relationships` |
| `PUT` | `/beacon/api/data-sources/{dataSourceId}/relationships/{relationshipId}` |
| `POST` | `/beacon/api/data-sources/{dataSourceId}/relationships/{relationshipId}/verify` |
| `DELETE` | `/beacon/api/data-sources/{dataSourceId}/relationships/{relationshipId}` |
| `POST` | `/beacon/api/data-sources/{dataSourceId}/relationships/discover-preview` |
| `GET` | `/beacon/api/data-sources/{dataSourceId}/schema-health` |

## Business glossary

The glossary teaches Beacon your organisation's vocabulary: what "active customer", "churn", or "net revenue" means in *your* schema. Terms are project-scoped and optionally narrowed to a single data source.

| Field | Purpose |
|---|---|
| `Term` | The business term as people say it |
| `Synonyms` | Alternative phrasings that should match the same term |
| `Definition` | Plain-language definition shown to the model |
| `TargetSchema` / `TargetTable` / `TargetColumn` | Where the term lives in the schema |
| `MetricExpression` | The SQL expression that computes it, when the term is a metric |
| `IsActive` | Retire a term without deleting its history |

When a question is asked, the top matching terms (`GlossaryTopK`, default 5) are injected into the **Business Glossary** block. Glossary retrieval is scoped to the caller's authorised project: two projects sharing a data source never see each other's definitions.

The glossary is managed through the API only and is gated with the Admin policy; there is no UI page for it.

| Method | Endpoint |
|---|---|
| `GET` | `/beacon/api/glossary?projectId={id}&dataSourceId={id}&includeInactive={bool}` |
| `POST` | `/beacon/api/glossary` |
| `PUT` | `/beacon/api/glossary/{id}` |
| `DELETE` | `/beacon/api/glossary/{id}` |

## Golden examples and the eval harness

A **golden case** is a human-verified question → SQL pair. Golden cases do double duty: they are injected into the grounding context as *authoritative* examples, and they are the regression suite the eval harness scores generation against.

### How cases are created

Only an Admin creates or edits a golden case. Feedback never does: rating an `ask` answer (through the `feedback` MCP tool, the MCP Playground, or `POST /beacon/api/eval/feedback`, which is Admins only) records the verdict, the user who recorded it and any checked corrected SQL on the answer's signal, and nothing else. The verdict is recorded whatever becomes of the corrected SQL: when the project does not retain query content, when the answer did not run on a single data source (a cross-source or knowledge answer: there is no dialect to check the SQL in), or when the [grounding SQL check](#who-curates-grounding) refuses it, only the SQL is dropped, and the answer says so (`correctedSqlRecorded: false` with `correctedSqlNotRecordedReason`).

- **Golden candidates.** `GET /beacon/api/eval/golden/candidates?projectId={id}` lists the answers rated `correct` that were not promoted yet. Only `ask` signals count, because a `query` or `dry_run` signal's "question" is the SQL it ran. The list is sorted newest first, and it is paged (`page`, `pageSize`, `sort`). Each candidate shows the question, the SQL a promotion would start from (the rater's corrected SQL, else the signal's own corrected SQL, else its generated SQL) with its `sqlHash` (the SHA-256 of that SQL, lowercase hex), the note, who asked and who rated it. Only signals of one project and one data source whose content was retained are listed. A project under the [content lock](https://moberghr.github.io/beacon/mcp/server/#content-retention) keeps no question to promote.
- **By promotion.** `POST /beacon/api/eval/golden/promote` takes the candidate's `signalId`, its `sqlHash` as `reviewedSqlHash`, and optional `notes`, and records the promoting user (`promotedBy`). A promotion applies to what the Admin reviewed: the signal must still be rated `correct` and still hold the SQL whose hash was sent, or the promotion is refused (`409`, "The answer changed since it was reviewed": reload the candidates and review it again). Only an answer to a question becomes a case (`400` for a `query` or `dry_run` signal); a missing signal, one without SQL, project or data source, a data source its project does not link, a question holding characters a reviewer cannot see, and SQL the check refuses answer `400` too, and nothing is created. The promotion claims its signal (`promotedAt`, `promotedEvalCaseId`) in the transaction that creates the case, only while the signal still holds the reviewed SQL (compared by the digest stored with the signal's SQL and written with every change to it, which is exact under any database collation; a signal recorded before Beacon kept the digest gets it first, only while it holds none and the SQL read). A signal is therefore promoted once: promoting it again, or two Admins promoting it at the same moment, answers with the case already promoted from it (`alreadyPromoted: true`, active or not) and creates nothing; a signal promoted before Beacon recorded promotions answers with the first case promoted from it.
- **By hand.** `PUT /beacon/api/eval/golden/{id}` edits the question, the gold SQL, the notes, or deactivates the case, and records the editing user (`modifiedBy`). A question holding characters a reviewer cannot see, or gold SQL the check refuses, is refused (`400`) and nothing changes.

Promoting and editing need a signed-in Beacon user (`403` otherwise): who did it is recorded. On the **MCP Learning** page an Admin sees three more tabs: **Golden candidates** (filtered by project, paged, each candidate in full with **Promote**; a candidate promoted meanwhile shows its existing case, and one that changed since it was listed says so; reload and review it again), **Golden examples** (a project's cases, showing who promoted and who last edited each, or *Not recorded* for a case created before Beacon recorded it; which cases a prompt leaves out; and **Deactivate**) and **Grounding recheck** (see [Rechecking stored grounding](#rechecking-stored-grounding)).

Gold SQL, promoted or edited, and corrected SQL from feedback are stored as the grounding SQL check returns them: without their comments. `GET /beacon/api/eval/golden` returns `promotedByUserId` and `modifiedByUserId` (`Users.Id`) with `promotedByName` and `modifiedByName` (the display name, or "User {id}"); all are empty for a case created before this was recorded. It also returns `goldSqlWithheld` and `questionWithheld`: whether a prompt leaves the case out.

### Injection

The top `GoldenExemplarTopK` (default 4) cases matching the question are rendered into **Verified query examples (authoritative)**, ranked *above* mined learned patterns and capped by `GoldenExemplarBudgetChars` (default 2000). Retrieval hard-filters on the caller's project, so golden SQL never leaks across project boundaries on a shared data source: each golden case's vector carries its project, and the nearest-neighbour search takes the top cases of the caller's project only, so another project's cases on the same data source never take its slots. Vectors indexed before they carried a project are stamped with their case's project by the `BackfillGoldenCaseEmbeddingProjects` database migration, without re-embedding them, so golden examples keep matching after the upgrade. Gold SQL is rendered without its comments and the question on one line; a case whose comments cannot be removed, or whose SQL or question holds characters a reviewer cannot see, is left out, and its id is logged once per hour.

### Eval runs

`POST /beacon/api/eval/runs` replays every active golden case for a project through the real generation pipeline and scores the result. Comparison is by **result-set fingerprint**. The generated SQL does not have to match the gold text; it has to produce the same rows. Each failure is tagged so that a user can distinguish a retrieval failure (the model never saw the right table) from a SQL-reasoning failure from an execution failure.

With `EnableEvalJudge` turned on, an opt-in LLM judge reviews a capped, PII-redacted sample of the rows to distinguish a cosmetically different result from a wrong one. It is off by default.

Both the gold SQL and the generated SQL are re-validated read-only and run through the query guardrail before execution. The harness inherits nothing and introduces no new execution path.

| Method | Endpoint |
|---|---|
| `POST` | `/beacon/api/eval/runs` |
| `GET` | `/beacon/api/eval/runs?projectId={id}&take={n}` |
| `GET` | `/beacon/api/eval/runs/{runId}/results` |
| `GET` | `/beacon/api/eval/golden?projectId={id}&dataSourceId={id}` |
| `GET` | `/beacon/api/eval/golden/candidates?projectId={id}` |
| `GET` | `/beacon/api/eval/grounding/recheck?projectId={id}` |
| `PUT` | `/beacon/api/eval/golden/{id}` |
| `POST` | `/beacon/api/eval/golden/promote` |
| `POST` | `/beacon/api/eval/feedback` |

The eval group is Admin-gated: it executes SQL against live data sources and writes golden cases.

## The learning loop

Every `ask`, `query`, and `dry_run` call records a **usage signal** holding the question, the generated SQL, the tables and columns referenced, the routing decision, the outcome and timings. A recurring job turns those signals into reusable knowledge.

![MCP Learning](https://moberghr.github.io/beacon/img/screenshots/mcp-learning-dark.png)

### From signal to injected lesson

1. **Mining.** Signals are clustered per project and data source. Only `ask` and `query` signals are mined: `dry_run` rows carry SQL rather than a natural-language question and are excluded so they cannot skew NL pattern mining or documentation-gap error rates. They remain in the signals table for analytics. A pattern's example question comes only from an `ask` signal (a `query` signal's "question" is the SQL it ran), on one line, and is left out when it holds characters a reviewer cannot see. Verdicts weigh in as follows: an `incorrect` verdict keeps the answer out of the common-query and join patterns; a `correct` verdict makes the answer the preferred example of its pattern and raises the pattern's confidence only when a [curator](#who-curates-grounding) recorded it. Where a verdict came from is recorded apart from who it is attributed to: the signal keeps the rating user (`FeedbackByUserId`) and, separately, whether the verdict came from a curator (`FeedbackFromCurator`), true only when an interactive session holding Curate in the signal's project at that moment recorded it (a project admin member or an Admin, signed in). A verdict from an API key or an MCP caller is attributed to the user it acts for and never weighs as a curator's, whatever that user's roles; neither does an Editor's or the asker's own, nor a verdict recorded before Beacon recorded its provenance. Mining reads the recorded provenance, not the rater's current roles.
2. **Lesson extraction.** For each failure-then-correction cluster, an LLM is asked to diagnose the failure and emit one compact, durable lesson. Only the cluster's text fields are sent, never result rows. If the provider is unavailable or the response does not parse, a deterministic regex + column-similarity path produces the lesson instead, so the loop never silently stops.
3. **Classification.** Lessons become learned patterns of type `SchemaCorrection`, `ColumnClarification`, `JoinPattern`, `CommonQuery`, `BusinessTermMapping`, or `DocumentationGap`.
4. **Review.** A new pattern lands as `NeedsEvidence`; confidence alone never approves anything. On the **MCP Learning** page (`/beacon/mcp-learning`) a [curator](#who-curates-grounding) of the pattern's project approves, rejects or sets it back to pending. These are the only statuses a person sets (`PUT /beacon/api/mcp/learned-patterns/{id}/status` answers `400` for any other; `AutoApproved` and `NeedsEvidence` are the job's). A decision applies to the content the reviewer saw: the list hands out each pattern's `contentHash` (the SHA-256 of its content, example question and example SQL), the status change sends it back as `reviewedContentHash`, and a pattern whose content changed since (also when the job refreshes it between the decision's read and its write) is not decided (`409`: reload the patterns and review it again). The same digest is stored with the pattern and written with every change to its content, and the decision's conditional write compares the stored digest. Fixed-length lowercase hex compares exactly under any database collation, where text equality on a case-insensitive SQL Server would let a concurrent case-only or trailing-space change pass. A pattern stored before Beacon kept the digest gets it on its first decision or auto-approval, stored only while the pattern holds no digest and the content read. The reviewer recorded is the caller (`403` for a caller without a Beacon user). The page shows each pattern in full with who reviewed it (*Unknown reviewer* for a review time without a recorded reviewer) and which examples a prompt leaves out; anyone else sees the reason instead of the actions. A pattern's example SQL passes the [grounding SQL check](#who-curates-grounding) when it is mined (SQL that fails is not stored: a new pattern gets none, an existing one keeps its example, and the run logs how many) and again when a person approves the pattern: approval stores it without comments, or is refused with the reason (`400`). Approving a pattern without example SQL checks nothing.
5. **Replay verification.** Auto-approval is decided by replay alone. With `EnableReplayVerification` on (the default), a `NeedsEvidence` candidate is *replayed*: Beacon re-runs the project's relevant golden cases with and without the candidate injected and auto-approves it only if it flips at least `LearningReplayMinFlips` (default 1) case from fail to pass with no regression. Replay is scoped to the candidate's own project and data source. Before it, the candidate's example SQL passes the grounding SQL check again and is replayed without its comments; a candidate whose SQL fails is not replayed and stays `NeedsEvidence`, and the run logs how many. The auto-approval is written only while the candidate is still `NeedsEvidence` and still holds the content the replay measured, compared by its stored digest as a person's decision is. With replay off, nothing is auto-approved: candidates wait for a person.
6. **Injection.** Approved and auto-approved patterns are ranked and injected into **Learned Patterns (from usage)** under `LearningInjectionBudgetChars` (default 1500), filtered to the caller's authorised project. Example SQL is rendered without its comments, read in the pattern's data source's dialect, and the example question on one line; an example that cannot be shown that way is left out and the pattern's content kept.

### Approval binds to the reviewed content

The aggregation refresh never rewrites the content, example question or example SQL of a pattern a person approved or rejected. Only its signal count and confidence refresh, so what was reviewed is what is injected. An auto-approved pattern whose content changes goes back to `NeedsEvidence` and is replayed again; the run logs how many went back. Content is compared exactly but for the statistics the job writes into its own sentences (the count of a `CommonQuery` ("… (12 similar queries)") or a `JoinPattern` ("… (8 queries)"), the error rate and query count of a `DocumentationGap` ("… a 40% error rate across 40 queries")), which the pattern's signal count and confidence also hold; any other content, a lesson's numbers included, is compared as written. The question is compared on one line, and the example SQL exactly but for its comments and the whitespace between its tokens, read by the data source's dialect: a string literal, a quoted identifier, a number and an identifier's case compare as written. So a count that moves, or SQL stored with comments before they were removed, is no change: the stored count refreshes and the pattern stays auto-approved, while a changed number in a lesson, a changed literal or an identifier in another case sends it back for replay. Every write the job makes to an existing pattern is conditional on the status it read, so a person's decision between the job's read and its write is never overwritten.

### Documentation patches

The learning job does not generate **documentation patches** today: no scheduled job turns approved schema corrections and documentation gaps into a proposed column or table description (or a section of the project's documentation). A patch that is already stored is still reviewed on the **Doc patches** tab, and applying or rejecting one is curation: a section patch needs Curate in its project; a description patch changes the data source's shared metadata, so it needs Curate in every live project linking the source, or a DataSourceAdmin (see [Who curates grounding](#who-curates-grounding)). This is deliberate: a description patch is mined from one project's signals, but the description it writes is what every project linking the source sees, so a project admin of the mining project alone does not apply it. The **Doc patches** tab shows each patch's current and proposed content in full and who applied or rejected it. Patches are scoped project by project: a project's patches came only from its own approved patterns, only for the data sources it linked, and only its own proposed or applied patches stopped a duplicate, so a data source shared by two projects never carries one project's patterns into the other project's patches. This describes patches already stored; no scheduled job generates new ones.

A description patch records the description it was proposed on and applies only while the column or table still holds exactly that text. A patch whose target changed since it was proposed, edited by hand or by another project's applied patch on a shared data source, is refused (`409`) and stays proposed; nothing is written and no event is recorded. Reject it; no scheduled job proposes a replacement on the current description. Section patches append to the project's own documentation and are not checked this way.

### Decay

A schema correction points at a specific column. When a schema refresh shows that column no longer exists, the correction is marked superseded and is never injected again. The row is kept for audit rather than deleted.

### Retention

Signals older than `LearningSignalRetentionDays` (default 90) are removed by the nightly cleanup job. Each project's signals use that project's effective window, and signals recorded without a project use the global one. The cleanup runs whether learning is on or off, so signals recorded while a project's own settings turn learning on still expire when the global switch is off.

The window is at least one day: a save with a lower value, globally or for a project, is rejected with `400` (a project's empty value inherits the global window). A window below one day stored before this check is never read as "delete every signal": a project's falls back to the global window, and the global one to the default 90 days. When one project's signals cannot be cleaned up, the job logs the project id and the failure's type, still cleans up every other project and the signals without a project, and then fails, so the run is recorded as failed rather than successful.

## Semantic retrieval

Keyword matching alone misses the question that says "revenue" when the column is called `net_amount`. Beacon adds a dense retrieval arm and fuses the two.

### Local embeddings

Embeddings are produced **in-process by a local ONNX model**: a 384-dimension bge-small-en-v1.5 with a WordPiece tokenizer and 512-token inputs. There is no network egress and no third-party embedding API. The model and tokenizer are local files you supply. The ONNX support is a separate package, `Moberg.Beacon.AI.Embeddings.Onnx`: add it and call `AddBeaconOnnxEmbeddings()` on the service collection. Embeddings are **disabled by default** so an install without the package or a model keeps working on the lexical path.

```json
{
  "Beacon": {
    "Embeddings": {
      "Enabled": true,
      "ModelPath": "/opt/beacon/models/bge-small-en-v1.5.onnx",
      "TokenizerPath": "/opt/beacon/models/vocab.txt"
    }
  }
}
```

On PostgreSQL, vectors are additionally written to a **pgvector** column with an HNSW cosine index so nearest-neighbour search runs in the database. This requires the pgvector extension to be available on the server, since the migration that adds the column runs `CREATE EXTENSION IF NOT EXISTS vector`. On SQL Server the same vectors are compared in memory from their byte representation, with no extension required. Semantic retrieval works on both; PostgreSQL is just faster at it.

### Fusion

Where a dense arm is available, its ranked results are merged with the keyword results using **reciprocal rank fusion**, with deterministic tie-breaks so paging stays stable. This drives:

- The `search` MCP tool across tables, columns, and documentation.
- Exemplar selection for golden examples and learned patterns (`ExemplarTopK`, default 4). A learned pattern is indexed only while its project links the data source and is not archived; its vector, like a golden case's, carries its own project, and selection filters on the caller's project, so a shared data source never surfaces another project's patterns or cases.
- Documentation chunk retrieval (`DocChunkTopK`, default 6).

Every dense path **fails open**: with no embedder configured, or with rows not yet indexed, retrieval falls back to the keyword/overlap path and returns results rather than failing.

### Documentation chunking

Generated project documentation is split into overlapping sentence windows (`DocChunkWindowSentences`, default 5; `DocChunkOverlapSentences`, default 1) and embedded so a question can retrieve the paragraph that answers it rather than a whole document. With `EnableContextualRetrieval` enabled, each chunk is additionally given a short LLM-written blurb situating it in its parent document before embedding, which gives more accurate retrieval at the cost of one LLM call per chunk at index time.

Fusion for documentation is at **section** granularity: two chunks of the same section collapse into one result.

## Background jobs

Indexing and aggregation run as recurring [Warp](https://moberghr.github.io/warp/) jobs in the host, visible on the `/warp` dashboard:

| Job | Schedule | What it does |
|---|---|---|
| `mcp-learning-aggregate` | Every 6 hours | Mines signals into learned patterns, runs decay and replay verification |
| `mcp-learning-cleanup` | Daily at 03:00 | Removes signals past the retention window |
| `mcp-embedding-reindex` | Every 12 hours | Re-embeds schema metadata and exemplars |
| `mcp-docchunk-reindex` | Every 12 hours | Re-chunks and re-embeds project documentation, each project under its own effective settings (a project that turned semantic or contextual retrieval off is skipped or gets no blurbs) |
| `mcp-audit-retention` | Daily at 03:30 | Purges MCP audit rows older than `Beacon:Mcp:Audit:RetentionDays` (a no-op while it is unset); see [MCP observability](https://moberghr.github.io/beacon/mcp/observability-and-audit/#retention) |

These are the sample host's registrations (`Beacon.SampleProject/Program.cs`); a host that embeds Beacon schedules them itself.

## Settings reference

These are the grounding, retrieval and learning settings of `McpSettingsData`; they round-trip through `GET`/`PUT /beacon/api/mcp/settings`. The guardrail settings (max row limit, read-only enforcement, PII detection and patterns, content retention) are described in [MCP Server → Configuration](https://moberghr.github.io/beacon/mcp/server/#configuration). The **MCP Settings** page (`/beacon/mcp-settings`) shows sample-value collection and value grounding (globally and per project) and the learning switch, auto-approve threshold, injection budget and signal retention. Everything else in this table is set through the API only; a save on the page sends every setting back as it read it (the stored values, `stored` on the `GET`), so the API-only ones keep their values.

| Setting | Default | Effect |
|---|---|---|
| `EnableSampleValueCollection` | `false` | Collect real sample values for the M-Schema block (and discover column values for documentation) |
| `EnableValueGrounding` | `false` | Probe live values for the literal-looking words of a question and add the matches to the context |
| `ValueGroundingMaxProbes` | `12` | Most value probes per question; on a shared data source the lowest value among the linking projects wins. Each probe has 5 seconds from its start (a slower probe matches nothing; the others keep their matches), and all the probes of one question stop at 15 seconds. They run one at a time, or up to three at a time while the caller's credential has `MaxConcurrentQueriesPerKey` slots free: each probe beside the call counts as one more query of that credential |
| `EnableLearning` | `true` | Master switch for signal mining |
| `LearningAutoApproveThreshold` | `0.7` | Stored with the settings; promotion is decided by replay verification (`EnableReplayVerification`, `LearningReplayMinFlips`) |
| `LearningInjectionBudgetChars` | `1500` | Character budget for the learned-patterns block |
| `LearningSignalRetentionDays` | `90` | Signal retention window, in days; at least 1 |
| `EnableReplayVerification` | `true` | Auto-approve a `NeedsEvidence` candidate only when golden-case replay shows it helps; off, nothing is auto-approved |
| `LearningReplayMinFlips` | `1` | Fail→pass flips a candidate must produce to be promoted |
| `EnableSemanticRetrieval` | `true` | Use the dense arm when an embedder is available |
| `ExemplarTopK` | `4` | Exemplars selected per question |
| `EnableGoldenExemplars` | `true` | Inject human-verified examples |
| `GoldenExemplarTopK` | `4` | Golden cases injected |
| `GoldenExemplarBudgetChars` | `2000` | Character budget for the golden-examples block |
| `GlossaryTopK` | `5` | Glossary terms injected |
| `EnableContextualRetrieval` | `false` | LLM-written situating blurb per doc chunk at index time |
| `DocChunkWindowSentences` | `5` | Sentences per documentation chunk |
| `DocChunkOverlapSentences` | `1` | Sentence overlap between chunks |
| `DocChunkTopK` | `6` | Documentation chunks retrieved |
| `EnableSelfConsistency` | `false` | Generate several SQL candidates and vote |
| `SelfConsistencyCandidateCount` | `5` | Candidates generated when voting is on |
| `SelfConsistencyMinTables` | `2` | Voting runs only for questions whose context spans at least this many tables |
| `EnableSemanticLint` | `true` | Deterministic checks on generated SQL in `ask` (an undeclared join, an aggregate fanning out over a one-to-many join, a non-aggregated column missing from `GROUP BY`), with one repair attempt from the shared repair budget |
| `EnableEvalJudge` | `false` | LLM judge adjudicates near-miss eval results |

## Project isolation

Grounding is retrieval, and retrieval is not authorisation, so every retrieval path is explicitly scoped to the project the caller is authorised for, not to the data source. On a data source shared by several projects, the glossary terms, golden examples, learned patterns, and replay cases one project sees are only ever its own.

The same holds for every grounding call that names a data source, and for every host that calls Beacon's AI services directly: each takes the caller's authorized project (`projectId`) and reads the data source only through it.

- **Project material** (DataModel documentation, code references from the project's repositories, glossary terms, golden examples and learned patterns) comes from that project only, never from another project that links the same data source.
- **A data source the project does not link** is not searched, described or catalogued: the table and column search finds nothing in it, the LLM context, table knowledge, lineage, data source overview and table schemas for it answer "not found", its schema catalog is empty (so every table in SQL against it fails the schema check), and value grounding probes nothing.
- **Live value probes need Query.** Value grounding probes the live data only for a caller who may query the project; a caller who may only read it (`get_query_context` as a Viewer) is grounded with the stored sample values alone.
- **The stored catalog is cached per project and data source.** `ask`, `get_query_context`, `query` and `dry_run` read a data source's stored tables and columns from a snapshot kept for up to 15 minutes, one per project. The project's link is checked on every call before the snapshot is read, and the stored sample values are screened under the project's settings on every call. A metadata refresh, a host model sync, an applied description patch, and a data source update, enable, disable, delete or migration-write mark drop the source's snapshots at once on the instance that made the change; another instance serves its own snapshot until it expires, including sample values that the change cleared (an archive, a change of target or a migration-write mark). The data source itself is refused on every instance at once when it is archived, disabled or marked as a write target.
- **Code references** belong to the repository that holds them, and a repository to one project: a scan names its project.
- An eval run needs a project (`400` without one).

## See Also

- [MCP Server](https://moberghr.github.io/beacon/mcp/server/): the tools that consume this context, and the guardrails around them
- [AI Integration](https://moberghr.github.io/beacon/features/ai-integration/): documentation generation and the LLM provider abstraction
- [Data Sources](https://moberghr.github.io/beacon/features/data-sources/): schema metadata loading
- [Admin Settings](https://moberghr.github.io/beacon/security/admin-settings/): LLM provider configuration