# Beacon: full documentation > Every page of the Beacon documentation in one file, in navigation order. Published at https://moberghr.github.io/beacon/. Every page is also available as Markdown at .md. --- # Overview > What Beacon is, who it is for, the ways to run it, and what you need before you start. Source: https://moberghr.github.io/beacon/getting-started/ Beacon watches your databases and tells people when something is wrong. You write SQL against the databases you care about, schedule it, and Beacon delivers the rows to email, Microsoft Teams, Slack, Jira or a webhook. The same governed access to your data is offered to AI assistants through a built-in [MCP](https://modelcontextprotocol.io) server. Beacon is a set of .NET 10 libraries with a React UI. You either run the sample host from the repository, or embed the NuGet packages in your own ASP.NET Core application. Since 5.0, Beacon can also be the [MCP gateway](https://moberghr.github.io/beacon/mcp/gateway/) of your own application: AI clients act as your users through your own endpoints, with no data sources at all ([what's new](https://moberghr.github.io/beacon/getting-started/whats-new/)). :::tip[Working with a coding agent?] [Build with AI](https://moberghr.github.io/beacon/getting-started/with-ai/) has a ready prompt for each goal, the one-file documentation bundles and the Claude Code plugin. The [cookbooks](https://moberghr.github.io/beacon/cookbooks/) take one task at a time from start to a verified result. ::: ## What it does | Capability | What you get | |---|---| | Database monitoring | Queries with one or more steps across nine connectors: PostgreSQL, SQL Server, MySQL, Azure Synapse, Snowflake, BigQuery, Databricks, AWS CloudWatch and REST APIs (OpenAPI). | | Alerting | Cron-scheduled subscriptions, notification triggers, tasks that stay open while a problem lasts, anomaly detection, data contracts, and a Control Tower view of their health. | | Data movement | Data migration jobs that copy data into a database an Admin marked as a write target. Off unless the deployment turns it on. | | Governed MCP server | `/beacon/mcp` for Claude, Cursor, VS Code and other MCP clients: eight built-in tools, saved queries published as tools, and optionally your own HTTP endpoints. SQL runs read-only, every call is scoped to the caller's projects and audited. See [MCP](https://moberghr.github.io/beacon/mcp/). | | MCP gateway | `Moberg.Beacon.Gateway` makes your own application an MCP server: your users sign in with Entra, your endpoints become tools that run as them, your documents ground the answers. | ## Who it is for - **Data and platform teams** who want SQL-based alerts on production data without writing a service for each one. - **Teams in regulated environments** who need read-only execution, project-level access control, four-eyes approvals and an audit trail. - **.NET teams** who want to give AI assistants access to their data through one governed endpoint instead of database credentials. ## Three ways to run Beacon | Path | Use it when | Start here | |---|---|---| | Run the sample host | You want to evaluate Beacon, or contribute to it. Clone the repository and run `Beacon.SampleProject`. | [Quick start](https://moberghr.github.io/beacon/getting-started/quick-start/) | | Embed the packages | You run Beacon for real. Add the `Moberg.Beacon.*` packages to your own ASP.NET Core application, with your sign-in, job runner and deployment. | [Integrate Beacon](https://moberghr.github.io/beacon/integrate/) | | MCP gateway | You want AI clients to use your own application, not your databases. One package and about thirty lines of `Program.cs`. | [MCP gateway for your application](https://moberghr.github.io/beacon/mcp/gateway/) | The sample host is a reference composition, not a product. Production deployments embed Beacon in their own host. ## How the pieces fit ```mermaid flowchart LR Browser["Browser"] Client["MCP client
(Claude, Cursor, VS Code)"] subgraph Host["ASP.NET Core host"] UI["React UI
/beacon"] API["REST API + SignalR
/beacon/api"] MCP["MCP server
/beacon/mcp"] Core["Beacon.Core
handlers, SQL gate, jobs"] end Meta[("Metadata database
PostgreSQL or SQL Server
schema beacon")] Sources[("Monitored data sources
read-only logins")] Notify["Email, Teams, Slack,
Jira, webhook"] LLM["LLM provider
(optional)"] Browser --> UI Browser --> API Client --> MCP API --> Core MCP --> Core Core --> Meta Core --> Sources Core --> Notify Core --> LLM ``` - The UI, REST API and MCP server all live under `/beacon`. The site root `/` belongs to the host. - Beacon keeps its own data (data sources, queries, users, audit) in the `beacon` schema of a PostgreSQL or SQL Server database. - Connection strings to monitored databases are encrypted at rest with `Beacon:EncryptionKey`. - Background work (scheduled subscriptions, re-indexing) runs on a job runner the host provides. The sample host uses [Moberg Warp](https://moberghr.github.io/warp/). ## Requirements | You need | Notes | |---|---| | .NET 10 SDK | Every Beacon project targets `net10.0`. | | A database for Beacon's own data | PostgreSQL with the [pgvector](https://github.com/pgvector/pgvector) extension available on the server, or SQL Server. | | PostgreSQL for the sample's job runner | The sample host runs Warp, which needs PostgreSQL even when Beacon's own data is on SQL Server. | | Node.js 22 and npm 11 | Only to build or run the React UI from source. Package consumers get the built UI inside `Moberg.Beacon.UI`. | | An encryption key | Base64 of at least 32 random bytes, for example from `openssl rand -base64 32`. Beacon does not start without one. | ## Where next - [Build with AI](https://moberghr.github.io/beacon/getting-started/with-ai/): prompts, bundles and the Claude Code plugin. - [Cookbooks](https://moberghr.github.io/beacon/cookbooks/): recipes for one task each, start to finish. - [What's new in 5.0](https://moberghr.github.io/beacon/getting-started/whats-new/): the gateway, and the upgrade checklist from 4.x. - [Core concepts](https://moberghr.github.io/beacon/getting-started/concepts/): data sources, projects, queries, subscriptions and the rest of the model. - [Quick start](https://moberghr.github.io/beacon/getting-started/quick-start/): run the sample host and send your first alert. - [Run the sample host](https://moberghr.github.io/beacon/getting-started/installation/): what the sample includes and how to configure it. - [Integrate Beacon](https://moberghr.github.io/beacon/integrate/): embed the packages in your own application. - [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/): every configuration key. - [Routes & endpoints](https://moberghr.github.io/beacon/reference/routes/): every URL Beacon serves. --- # What's new in Beacon 5.0 > Beacon 5.0 makes Beacon the MCP gateway of your own ASP.NET Core application, adds test and verification tooling, moves LLM providers into packages and slims the dependency graph. What is new, and the checklist for upgrading from 4.x. Source: https://moberghr.github.io/beacon/getting-started/whats-new/ Beacon 5.0 was released on 21 October 2026 (NuGet version `5.0.0`). Its headline is the **MCP gateway**: one package that turns your own ASP.NET Core application into an MCP server, so Claude and other AI clients act as your signed-in users through your own endpoints. Around it come test and verification tooling, a Claude Code plugin, LLM providers as packages, and a lighter dependency graph. :::tip[Upgrading with a coding agent?] ```text wrap Upgrade this application from Beacon 4.x to 5.0. Read https://moberghr.github.io/beacon/getting-started/whats-new.md, check this repository against every item under "Upgrade from 4.x", apply the ones that apply, and tell me which ones did not apply and why. Build and run the tests after the change. ``` ::: ## Highlights ### MCP gateway for your own application `Moberg.Beacon.Gateway` embeds Beacon as the MCP server of an existing application. People connect Claude to `https://your-app/beacon/mcp` and sign in with Microsoft Entra. A tool call then runs one of your endpoints as that person, through your own authorization, and Beacon audits it. Your Markdown documents can ground the answers. There are no data sources and no SQL. Read-only enforcement is pinned, and so is no content retention unless you set `RetainToolContent`. `/beacon/mcp` answers 401/403 itself and never redirects to your sign-in. ```csharp builder.Services.AddBeaconMcpGateway(builder.Configuration, x => { x.ProjectName = "app"; x.Database(db => db.UsePostgreSql()); x.Entra(builder.Configuration.GetSection("AzureAd")); x.Users(); // your IMcpHostUserResolver: who a token is }); // … app.UseBeaconMcpGateway(); app.MapBeaconMcpGateway(); ``` An optional admin UI serves the MCP audit, the settings and your documents at `/beacon` to holders of one of your own authorization policies, signed in through your own sign-in. → [MCP gateway for your application](https://moberghr.github.io/beacon/mcp/gateway/) ### Endpoint tools, ready for real applications - `[BeaconTool]` works on handler classes whose framework copies class attributes into the endpoint, such as a Moberg Warp handler. On an MVC controller class it fails at startup and tells you to put the attribute on the action. - `BindFromCaller("userId", "app_user_id")` fills the person's own id from the caller and removes the argument from the tool's schema. An agent cannot pass someone else's id. - Tool calls run your `IExceptionHandler`s, so a refusal reaches the agent as your own problem details. - `httpContext.User.IsBeaconMcpCall()` tells you a request is an agent's. You can then hold agents to less than the browser. - `AddBeaconMcp(x => …)` chooses the built-in tools, turns saved-query tools off and replaces the server instructions. → [Host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/), recipe: [Expose an endpoint as a tool](https://moberghr.github.io/beacon/cookbooks/endpoint-as-tool/) ### Test it and verify it - **`Moberg.Beacon.Testing`** has Entra-shaped test tokens, an in-memory user resolver and a real MCP client over your test server (`McpTestTokens`, `McpTestUsers`, `McpTestClient`). Your tests then go through the bearer middleware, the caller mapping, the MCP filters, the audit and your endpoint's own authorization. - **`beacon-doctor`** (`dotnet tool install -g Moberg.Beacon.Doctor`) checks a deployed endpoint the way an MCP client sees it: the 401 challenge, the protected-resource metadata, the authorization server, PKCE and `offline_access`. With `--token` it also runs `initialize` and `tools/list`. It prints PASS/WARN/FAIL lines and exits non-zero on a failure. - `src/Beacon.SampleGateway` in the repository is a reference gateway host. ### Built for coding agents - A **Claude Code plugin** ships in the repository: `/plugin marketplace add moberghr/beacon`, then `/plugin install beacon@moberg-beacon`. → [Build with AI](https://moberghr.github.io/beacon/getting-started/with-ai/) - **Cookbooks**: recipes that run start to finish, each with a prompt you can hand to your agent. → [Cookbooks](https://moberghr.github.io/beacon/cookbooks/) - New one-file bundles: `llms-mcp.txt`, `llms-cookbooks.txt` and `llms-gateway.txt`, next to `llms-integrate.txt` and `llms-full.txt`. ### LLM providers as packages `AddBeaconAI` registers OpenAI, Azure OpenAI and Anthropic (Claude). **AWS Bedrock moved to its own package**, `Moberg.Beacon.AI.Bedrock`, which you activate with `AddBeaconBedrock()`, so `Moberg.Beacon.AI` no longer depends on the AWS SDK. The admin settings page lists only the providers you installed (`GET /beacon/api/admin-settings/llm-providers`). A configured provider that is not installed leaves AI without a provider: the host logs a warning, and AI calls fail with a message that names the package to add. **Local embeddings moved to their own package too**: `Moberg.Beacon.AI.Embeddings.Onnx`, activated with `AddBeaconOnnxEmbeddings()`. Without it, retrieval is keyword-based and `Moberg.Beacon.AI` carries no ONNX runtime. ### Migration writers in the connectors, and a lighter build Data-migration writes moved out of `Moberg.Beacon.Core` into the destination's connector, behind `IMigrationBulkWriter`. PostgreSQL writes with binary `COPY`, SQL Server with `SqlBulkCopy` and `MERGE`, MySQL with multi-row parameterised inserts, Upsert with `ON DUPLICATE KEY UPDATE`, and Truncate-Load. Every write, the clear and the primary key lookup run on the run's one connection and transaction, so a run is all-or-nothing on PostgreSQL and SQL Server, and an Insert or Upsert run is all-or-nothing on MySQL InnoDB. The exception is a MySQL Truncate-Load: `TRUNCATE` commits on its own, so a run that fails after the clear leaves the table empty. `EFCore.BulkExtensions` and `Pomelo.EntityFrameworkCore.MySql` have left the build, and `Moberg.Beacon.Core` no longer brings the SQL Server EF Core provider with it. ### Under the hood - Beacon's admin standing comes from the stored user's live roles, not from the session's role claim (see the upgrade list). Demoting someone takes effect on their next request. - `/beacon/mcp` accepts MCP credentials only: an API key or a mapped bearer token. - Every SQL gate request now carries a row-level security context. This is groundwork: 5.0 filters no rows, and the lock `Beacon:RowSecurity:Enabled` stays off (a host that sets it on does not start). ## New packages | Package | What it is | |---|---| | `Moberg.Beacon.Gateway` | The MCP gateway for your own application | | `Moberg.Beacon.Testing` | Test helpers for a gateway host's MCP tools | | `Moberg.Beacon.Doctor` | The `beacon-doctor` .NET tool | | `Moberg.Beacon.AI.Bedrock` | The AWS Bedrock LLM provider, previously inside `Moberg.Beacon.AI` | | `Moberg.Beacon.AI.Embeddings.Onnx` | Local ONNX embeddings for semantic retrieval, previously inside `Moberg.Beacon.AI` | ## Upgrade from 4.x Upgrade every `Moberg.Beacon.*` package to `5.0.0` together, then go through this list. Most hosts need only the first two items. - [ ] **Apply the new migration.** `AddRowSecurityModel` adds five tables in the `beacon` schema and a column on `data_sources`, on PostgreSQL and on SQL Server. A host that calls `Beacon.Core.ServiceConfiguration.UseBeacon(app.Services, createSchema: true)` applies it at startup. A host that migrates in a separate step can call `BeaconPostgreSqlMigrator.MigrateAsync(...)` or `BeaconSqlServerMigrator.MigrateAsync(...)` from it. → [Metadata database](https://moberghr.github.io/beacon/integrate/database/) - [ ] **Bedrock users: add the package.** Reference `Moberg.Beacon.AI.Bedrock` and call `services.AddBeaconBedrock()` (before or after `AddBeaconAI`). Without them, Bedrock is not available and AI calls fail with "LLM provider Bedrock is not installed". → [AI providers](https://moberghr.github.io/beacon/integrate/ai/) - [ ] **Semantic retrieval: add the embeddings package.** Reference `Moberg.Beacon.AI.Embeddings.Onnx` and call `services.AddBeaconOnnxEmbeddings()` (before or after `AddBeaconAI`). `AddBeaconEmbeddings()` is obsolete and no longer loads the model. Without the package, retrieval falls back to keywords; with `Beacon:Embeddings:Enabled` on, the host logs one warning the first time retrieval asks for embeddings. → [AI providers](https://moberghr.github.io/beacon/integrate/ai/) - [ ] **Migration targets: register the connector.** A data-migration run writes through the destination's connector, so a PostgreSQL, SQL Server or MySQL target needs that connector package and its `AddConnector()` call. A run into an engine without one fails before Beacon decrypts the target's credentials. If you run MySQL Truncate-Load jobs, keep a way to reload the table: a failed run leaves it empty. → [Data migration](https://moberghr.github.io/beacon/features/data-migration/) - [ ] **Central package management: raise your pins.** If restore fails with `NU1109` ("Detected package downgrade"), raise each pinned package to the version the message names (EF Core and `Microsoft.Extensions.*` patches, `ClosedXML`). → [Packages](https://moberghr.github.io/beacon/integrate/packages/) - [ ] **No browser sessions on `/beacon/mcp`.** A signed-in browser session, Beacon's or yours, now gets `403` there. MCP clients use an API key or a mapped Entra token. The MCP playground is not affected: it calls `/beacon/api`. → [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/) - [ ] **Hosts that own sign-in: grant Admin in Beacon.** While Beacon manages users, Admin comes from the Beacon user's own roles. An identity-provider role named `Admin` (an Entra app role, for example) no longer makes someone Beacon's Admin: grant the Admin role to their Beacon user. Functional permissions mapped with `Beacon:Authorization:IdpRoles` are unchanged. Hosts without Beacon user management keep the claim. → [User management](https://moberghr.github.io/beacon/security/user-management/) - [ ] **Hosts that use Core without the API or MCP packages:** the request principal must be registered. Call `AddBeaconApiServices`, `AddBeaconCookieAuthentication` or `AddBeaconMcp`, or register your own with `services.UseRequestPrincipal()`. `IHttpContextAccessor` alone no longer identifies the caller. → [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/) - [ ] **SQL Server provider no longer transitive.** `Moberg.Beacon.Core` no longer references `Microsoft.EntityFrameworkCore.SqlServer` or `EFCore.BulkExtensions`. A host on SQL Server already references `Moberg.Beacon.Core.SqlServer`. Add your own reference only if your own code used those packages through Beacon. The same goes for `AWSSDK.BedrockRuntime` and the ONNX packages, which `Moberg.Beacon.AI` no longer brings. - [ ] **Code that calls Beacon's internals:** - `SqlGateRequest` takes a `RowSecurity` argument, which you get from `IRowSecurityResolver`. - The `LlmProviderFactory(accessor, logger)` constructor is obsolete. Resolve the provider registrations from DI. - `Constants.Migration.MaxFailedRowsBeforeStop` is gone. - `IMigrationBulkContextProvider`, `MigrationBulkContextFactory` and `DynamicDbContext` are removed with `EFCore.BulkExtensions`, and `SqlServerDataTransferManager` and `ObjectDataReader` with the old SQL Server bulk path. Migration writes use the connectors' `IMigrationBulkWriter`. - `BedrockProvider` moved to the `Beacon.AI.Bedrock` assembly and `IHostEndpointDispatchMiddleware` to `Beacon.MCP`. Both keep their namespaces: recompile. - `SqlClientTransportInspector` has an abstract `Read(string)` method, so a class of your own that derives from it must implement it. - `DatabaseAuthorizationProvider`'s constructor takes `IBeaconRequestPrincipal`, and its `(IUserManagementService)` overload is gone. - `AuthenticationPathPrefixes.Matches(PathString)` is gone: use `BeaconRequestPaths.Matches`. ## Behaviour changes worth knowing | Area | In 5.0 | |---|---| | MySQL migration runs | All-or-nothing on InnoDB. A run no longer ends as a partial success. A Truncate-Load that fails after the clear leaves the table empty. Upsert matches on any unique key and no longer needs the `CREATE TEMPORARY TABLES` privilege. | | AI provider | Choosing a provider that needs an API key requires typing a key. Saving or testing a provider that is not installed is refused while AI is on. | | Admin standing | A user who loses the Admin role loses Admin routes on their next request, not at the next session revalidation. | | PostgreSQL and SQL Server migration runs | All-or-nothing: the writes and the key lookup share the run's transaction, with no row-by-row fallback. An Upsert into a table without a primary key fails. An unqualified SQL Server table means `dbo`. | | SQL Server migration Insert | Keeps source identity values and NULLs (writing identity values needs `ALTER` on the table) and checks CHECK and foreign-key constraints: a violating row fails the run. | | XLSX attachments | Dates keep a date format per cell, `DateOnly` and `DateTimeOffset` are written as dates, and a report with no rows is an empty workbook instead of an error. | ## After 5.0.0 These changes are on `main` and ship in the next release after 5.0.0. ### OpenTelemetry: the MCP SDK's spans need `AddBeaconMcpSdkTracing()` `AddBeaconInstrumentation()` on the tracer provider no longer subscribes to the MCP SDK's own `tools/call` spans (source `Experimental.ModelContextProtocol`); it adds only Beacon's `"Beacon"` source. A host that upgrades loses those spans until it also calls `AddBeaconMcpSdkTracing()` on the same builder, which adds the source and clears the status descriptions the SDK writes (they can carry an error result's content). Do not add `Experimental.ModelContextProtocol` yourself: that bypasses the scrubber. The meter extension is unchanged. ### OpenAI and Azure OpenAI over plain HTTP OpenAI and Azure OpenAI now go over plain HTTP, as Claude already did, so `Moberg.Beacon.AI` no longer brings the `OpenAI`, `Azure.AI.OpenAI` or `Azure.Core` packages. Requests keep the SDKs' wire format and retry policy (up to three retries on 408, 429 and 5xx, honouring `Retry-After` up to 60 seconds). Redirects are not followed, so an API key never travels to a redirect target. Failures name the HTTP status and the provider's error code, never the response body, and the request queue does not resend a call that failed with a status, since the provider already retried it. This also fixes Azure OpenAI, which failed on every call in 4.x and 5.0.0 with a `MissingMethodException`. When you upgrade: - [ ] **Packages no longer transitive.** If your own code used `OpenAI`, `Azure.AI.OpenAI` or `Azure.Core` through Beacon, reference them yourself. - [ ] **Code that builds providers itself:** `OpenAiProvider` and `AzureOpenAiProvider` keep their constructors but are obsolete. Providers are built through the `ILlmProviderRegistration`s that `AddBeaconAI()` installs, and a host's own registration for a built-in provider now wins whatever the call order. ### Parameter placeholders have a shape A step parameter's placeholder must be `{identifier}`: letters, digits and underscore, starting with a letter or underscore (so `{1}` is refused, although the editor's auto-detect accepts digits). An empty placeholder is refused, it must be distinct within the step, and a multi-step save now also requires every placeholder to occur in its step's SQL. Beacon checks it when a query is created or edited, when an AI actor proposes a change and when a query is newly exposed as an MCP tool. Stored queries keep running as they are, and restoring an old version keeps its placeholders as stored, also on an exposed tool. An already-exposed tool keeps running and can be edited or disabled; only newly exposing it needs the shape. The next edit of a stored query asks for a valid placeholder. When you upgrade: - [ ] **Hosts or scripts that create queries through the API:** send `{name}` placeholders. A bare word, a quoted or spaced placeholder is refused with the parameter's name. ### Stronger password hashes, checked user names Internal users' passwords are hashed at 600,000 PBKDF2 iterations, and the count is stored in the hash. Existing hashes keep working and are upgraded at each user's next successful password sign-in ([Password hashes](https://moberghr.github.io/beacon/security/user-management/#password-hashes)). Creating a user, and first-run setup, now trim the user name and e-mail address and refuse a user name outside 3 to 100 characters or with control characters, an e-mail address that is not one plain address, and (setup) a password confirmation that differs. Editing a user keeps a value sent back exactly as stored, and trims and checks only the fields that change. When you upgrade: - [ ] **Rollback:** an earlier version cannot read an upgraded hash. After a rollback, reset the passwords of internal users who signed in on this version. - [ ] **Scripts that create users through the API:** send a user name of at least 3 characters and a plain e-mail address (or none). ### Security headers and an origin check on Beacon's routes Beacon's responses now carry `X-Content-Type-Options`, `X-Frame-Options: DENY`, `Referrer-Policy: strict-origin` and a `Permissions-Policy`, and responses under `/beacon` a Content-Security-Policy that allows scripts, styles and workers from Beacon's own origin only and no framing. A new origin check looks at state-changing, realtime-hub and anonymous Beacon requests and, in this release, only logs the ones from a foreign origin. `UseBeaconAuthentication()` adds both; your own routes are untouched. See [Security headers and the origin check](https://moberghr.github.io/beacon/integrate/host-setup/#security-headers-and-the-origin-check). When you upgrade: - [ ] **Hosts that frame Beacon or load it into another page's content:** `frame-ancestors 'none'` and `X-Frame-Options: DENY` now refuse it. Send your own policy on those responses, or set `Beacon:Security:Csp` to `ReportOnly`. - [ ] **Behind a proxy:** set `Beacon:PublicBaseUrl` (or `Beacon:Security:AllowedOrigins`), watch the log for the origin check's warnings, then set `Beacon:Security:OriginGuard` to `Enforce`. A later release is planned to make `Enforce` the default. - [ ] **Optional:** call `app.UseBeaconSecurityHeaders()` before `UseStaticFiles()` so the UI's files carry the headers too. ### Retention by default A new data retention job can keep run history and logs bounded (see [Data retention](https://moberghr.github.io/beacon/getting-started/configuration/#data-retention-beaconretention)): with `Beacon:Retention:Days` set it deletes rows of subscription runs, notifications, anomaly baselines and events, AI actor runs and plans, migration runs, MCP eval runs and the execution log older than the window (90 days recommended; the MCP audit's own window, 365 days recommended). `Beacon:Retention:ContentDays` (30 recommended for regulated deployments) clears the SQL text, result snapshots and model output of older rows, and `Beacon:Retention:Locked` refuses any window that keeps data forever. Every window defaults to keep forever. The content lock (`RetainQueryContent` off globally, `Beacon:Mcp:ForceNoContentRetention`) now also covers those stores outside MCP. When you upgrade: - [ ] **Nothing is deleted on upgrade.** Retention is off until you set a window; while none is set the host logs one Information at startup (`RetentionUnconfigured`). Set the recommended windows to start purging. - [ ] **Schedule the job.** Hosts with their own scheduler call `IDataRetentionService.RunAsync` daily; the sample host runs it as the Warp job `data-retention`. - [ ] **Content lock wider.** With `RetainQueryContent` off globally or `ForceNoContentRetention` on, subscription result snapshots, the SQL text of runs and of the execution log, AI actor output and migration statements are no longer stored, and result snapshots stored before are cleared by the retention job. ### Every declared foreign key becomes a relationship A column that takes part in more than one foreign key (for example `tenant_id`, referencing `tenants` on its own and `x` together with `x_id`) now yields a relationship for each of them, so the composite key keeps both of its column pairs. Before, Beacon kept one foreign key per column, and which one depended on the order the catalog returned. This applies to SQL Server, Azure Synapse, PostgreSQL and MySQL sources. The column's own foreign-key fields keep naming one target: the foreign key whose constraint name sorts first. When you upgrade: - [ ] **Data sources with overlapping foreign keys:** more foreign-key relationships appear after the next metadata refresh. Nothing changes until a source is refreshed, and no relationship is removed. ## Release notes The full list of changes is on the [5.0.0.0 release](https://github.com/moberghr/beacon/releases/tag/5.0.0.0). --- # Build with AI > Point Claude Code, Cursor or any coding agent at Beacon: copy-paste prompts for each goal, one-file documentation bundles, the Claude Code plugin, and a snippet that keeps Beacon in your agent's instructions. Source: https://moberghr.github.io/beacon/getting-started/with-ai/ Most people add Beacon to an application with a coding agent's help, and these docs are written for that. Every page is also served as plain Markdown, each section is bundled into one file, and the Claude Code plugin teaches the agent how to use them. Pick your goal, paste the prompt, and answer the questions the agent asks you. ## Pick your goal, copy the prompt **Give AI clients access to your own application** (your users, your endpoints as tools, no data sources): ```text wrap title="Paste into your coding agent" Add an MCP gateway to this ASP.NET Core app with Beacon. Read https://moberghr.github.io/beacon/llms-gateway.txt first and follow it step by step. Ask me before every decision it marks as one, and never put a secret in a committed file. ``` **Embed Beacon's data product** (monitoring, alerts, data movement, and an MCP server over your databases): ```text wrap title="Paste into your coding agent" Integrate Beacon into this ASP.NET Core app. Read https://moberghr.github.io/beacon/llms-integrate.txt first and follow its checklist step by step. Ask me before every decision the guide marks as one. ``` **Connect your AI client to a Beacon that is already running:** ```text wrap title="Paste into your coding agent" Connect this project's AI tooling to our Beacon MCP server at https://beacon.example.com/beacon/mcp. Read https://moberghr.github.io/beacon/cookbooks/connect-coding-agent.md and follow it. Read the API key from the BEACON_MCP_API_KEY environment variable; never write the key into a file. ``` **Do one specific task** (publish a saved query as a tool, single sign-on with Entra, alert Slack on a failed check, …): ```text wrap title="Paste into your coding agent" Read https://moberghr.github.io/beacon/llms-cookbooks.txt, pick the recipe for: , tell me which one you picked, then follow it step by step and show me each "Check it works" result. ``` **Upgrade from Beacon 4.x:** ```text wrap title="Paste into your coding agent" Upgrade this application from Beacon 4.x to 5.0. Read https://moberghr.github.io/beacon/getting-started/whats-new.md and apply every item under "Upgrade from 4.x" that applies to this repository. Build and run the tests afterwards. ``` More prompts, including the ones you say to Claude once it is connected to Beacon, are in the [prompt library](https://moberghr.github.io/beacon/cookbooks/prompts/). ## What your agent can read | File | Content | Use it for | |---|---|---| | [`llms.txt`](https://moberghr.github.io/beacon/llms.txt) | The index: every page with a one-line summary, and the bundles | Letting the agent find its own way | | [`llms-gateway.txt`](https://moberghr.github.io/beacon/llms-gateway.txt) | The MCP gateway guide and the pages it uses | An MCP server for your own application | | [`llms-integrate.txt`](https://moberghr.github.io/beacon/llms-integrate.txt) | The integration checklist and every page it links | Embedding Beacon's data product | | [`llms-mcp.txt`](https://moberghr.github.io/beacon/llms-mcp.txt) | The whole MCP section | Anything about the MCP server, its tools and clients | | [`llms-cookbooks.txt`](https://moberghr.github.io/beacon/llms-cookbooks.txt) | Every recipe and the prompt library | One specific task | | [`llms-full.txt`](https://moberghr.github.io/beacon/llms-full.txt) | Every page | Questions that span the product | Every page is also available as Markdown: add `.md` to its URL (for example `https://moberghr.github.io/beacon/mcp/gateway.md`), or use **Copy as Markdown** under the page title. A single page is the smallest context for a narrow question. A bundle saves the agent from fetching pages one by one. ## The Claude Code plugin Beacon's repository is a Claude Code plugin marketplace. Install it once: ```text wrap /plugin marketplace add moberghr/beacon /plugin install beacon@moberg-beacon ``` | Skill | Claude uses it when you ask to | |---|---| | `beacon-integrate` | Embed Beacon's data product in your application | | `beacon-gateway` | Add an MCP gateway to your application | | `beacon-tool` | Make one endpoint available to Claude as a tool, safely | | `beacon-entra` | Prepare the Microsoft Entra registrations for Claude's connector, or fix an `AADSTS` error | The skills follow the documentation above: they fetch the current guide instead of relying on what the model remembers. They read your application before deciding anything, ask the decisions the guide lists, and stop before anything outward-facing, such as changing your tenant, pushing or deploying. ## Keep Beacon in your agent's instructions Once Beacon is in your repository, add a short section to its `CLAUDE.md` or `AGENTS.md`, so every later session knows where Beacon's rules are: ```md title="CLAUDE.md / AGENTS.md" ## Beacon This application embeds Moberg Beacon 5.x (NuGet `Moberg.Beacon.*`, all at the same version). - Docs for agents: https://moberghr.github.io/beacon/llms.txt (pick the bundle for the task). Read the relevant page before changing anything Beacon wires: middleware order, authentication, MCP tools. - Secrets (`Beacon:EncryptionKey`, connection strings, API keys) live in user secrets or the secret store, never in a committed file. - An endpoint marked as an MCP tool (`WithBeaconTool` / `[BeaconTool]`) is self-service: it returns or changes only the signed-in person's own records, declares `ReadOnly`, has authorization metadata, and takes the person's id from the caller (`BindFromCaller`), never from the agent. Add a test with `Moberg.Beacon.Testing` for each one. - Beacon refuses to start on most wiring mistakes, and the message says the fix: read it before changing code. ``` ## Getting good results - **Let the agent ask.** The guides mark the choices that are yours, such as who may connect, which endpoints become tools, and where Beacon's tables live. A prompt that says "ask me before every decision" keeps the agent from guessing. - **Point it at the source of truth.** Beacon's documentation describes the current release. An agent's memory of Beacon may be older, and the plugin's skills fetch the docs for that reason. - **Use the reference hosts.** `src/Beacon.SampleProject/Program.cs` (the full product) and `src/Beacon.SampleGateway` (a gateway host) in the [repository](https://github.com/moberghr/beacon) are complete compositions to compare against. - **Make it prove it.** Every recipe ends with "Check it works". Ask the agent to show you each result, and to run `beacon-doctor` against a deployed MCP endpoint before you connect Claude to it. - **Keep secrets out of the chat.** Give the agent environment variable names, not API keys or client secrets. --- # Core concepts > The mental model behind Beacon: data sources, projects, queries, subscriptions, recipients, tasks, data contracts, approvals, the SQL execution gate, the MCP server, roles and audit. Source: https://moberghr.github.io/beacon/getting-started/concepts/ This page explains the objects Beacon works with and how they relate. Each section is short and links to the page with the details. ## How the objects relate ```mermaid flowchart LR Project["Project
(access boundary)"] DS1[("Data source")] DS2[("Data source")] Query["Query
(steps + versions)"] Sub["Subscription
(cron, owner, trigger)"] Rec["Recipient
(Email, Teams, Slack, Jira, webhook)"] Task["Task"] Tool["MCP tool
q_name"] Project -- links --> DS1 Project -- links --> DS2 Query -- "step 1 reads" --> DS1 Query -- "step 2 reads" --> DS2 Sub -- runs --> Query Sub -- notifies --> Rec Sub -- opens --> Task Query -. "published as" .-> Tool ``` A query is reachable only through a project that links every data source it reads. A subscription runs one query on a schedule and delivers the rows to its recipients. ## Data source A connection to a system Beacon reads: a database (PostgreSQL, SQL Server, MySQL, Azure Synapse, Snowflake), BigQuery, Databricks, AWS CloudWatch or a REST API described by OpenAPI. Each kind is served by a connector package. - The connection string is encrypted at rest with `Beacon:EncryptionKey`. It is never shown again after you save it. - Give every data source a **read-only database login**. Beacon also runs SQL read-only itself, and on PostgreSQL and MySQL inside a read-only transaction, but the login is the defence that holds on every engine. - Beacon loads **metadata** (schemas, tables, columns, relationships) to power the query editor, the MCP catalog and SQL grounding. - A new data source is usable only by Admins until it is linked into a project. See [Data sources](https://moberghr.github.io/beacon/features/data-sources/). ## Project The access boundary. A project links data sources and has members. - **Members** hold a project role: Viewer, Editor or ProjectAdmin. - **Restricted** projects are visible only to their members and Admins. Everyone else gets `404`. New projects are restricted and their creator becomes ProjectAdmin. - **Open** projects are visible to every user with a Beacon role, at the level of that role. - Queries, subscriptions, notifications, tasks and MCP calls are all filtered by the projects the caller can read. See [Authorization](https://moberghr.github.io/beacon/security/authorization/) and [Security model](https://moberghr.github.io/beacon/security/). ## Query Saved SQL, made of one or more **steps**. Each step runs on one data source. - A later step reads an earlier step's rows as `@result1`, `@result2`, and so on. Beacon loads intermediate results into an in-memory SQLite database and runs the next step there, so a query can join data from different engines. - `{name}` placeholders become **parameters**. Values are bound as database parameters, never pasted into the SQL. - Changes are kept as **versions** you can compare and restore. - A query can be **published as an MCP tool** (`q_`) by a SecurityAdmin. See [Queries](https://moberghr.github.io/beacon/features/queries/). ## Subscription A query on a schedule. - **Schedule**: a standard 5-field cron expression, for example `0 9 * * *` for 09:00 every day. - **Owner**: the user the scheduled run acts as. Before every run Beacon re-checks that the owner may still run the query; a refused run is recorded as skipped, never run as the system. - **Trigger**: when a run notifies: every execution, when the row count changes, or when it increases. A run that returns no rows never notifies. - **Results**: max rows in the message, an optional CSV or XLSX attachment, an optional copy of the query text, and stored result rows. - **Tasks**: optionally open a task while the query keeps returning rows. - **Anomaly detection**: optionally notify only when the row count is unusual compared with earlier runs. - **SLA**: an optional number of hours, shown on the subscription and its tasks. Subscriptions cannot be edited after creation, except recipients, SLA and owner. See [Subscriptions](https://moberghr.github.io/beacon/features/subscriptions/) and [Anomaly detection](https://moberghr.github.io/beacon/features/anomaly-detection/). ## Recipient and notification A **recipient** is a destination: Email, Microsoft Teams, Slack, Jira or a generic webhook. A **notification** is one delivery of one run's results, kept in the notification history with its status. - Only an Admin or a SecurityAdmin creates or changes recipients. - A **destination policy** decides where Beacon may deliver: HTTPS only, vendor hosts for Slack, Teams and Jira, public hosts for webhooks, and optional allow-lists and disabled channels set by the host (`Beacon:Notifications`). - Email needs an email adapter registered by the host. The sample host's adapter sends through SMTP (`Beacon:Smtp:*`). - Rows that reach recipients are PII-masked under the strictest settings of the projects involved. See [Notifications](https://moberghr.github.io/beacon/features/notifications/). ## Task A work item that opens when a subscription with task creation on returns rows, updates while later runs keep returning rows, and resolves itself when a run returns none. People assign, prioritise, snooze and comment on it. See [Tasks](https://moberghr.github.io/beacon/features/tasks/). ## Data contract A set of data-quality rules on one table (freshness, row count, uniqueness, nulls and more), evaluated on a cron schedule into a weighted score, with an alert when the score falls below a threshold. Contracts run on PostgreSQL, SQL Server and MySQL data sources. Only an Admin creates contracts that contain Custom SQL rules. See [Data quality](https://moberghr.github.io/beacon/features/data-quality/). ## Approvals An opt-in four-eyes workflow. When the host enables it, a change to a query's executable content, a migration job, a Custom SQL rule or a recipient's destination takes effect only after a second person with the Approver permission approves exactly the reviewed content. It is off by default. See [Approvals](https://moberghr.github.io/beacon/security/approvals/). ## The SQL execution gate Every statement Beacon runs on a data source passes one gate first, whoever asked for it: a subscription, a preview, a data contract or an MCP tool. The gate refuses anything but a single read-only statement, refuses names from other databases and SQL that switches context or runs dynamic SQL, and writes a row cap into the SQL. For subscriptions and previews, a result larger than `Beacon:Query:MaxExecutionRows` fails the run rather than being cut. Data migration, the only write path, is separate and off by default. See [Security model](https://moberghr.github.io/beacon/security/) and [Queries](https://moberghr.github.io/beacon/features/queries/). ## MCP server and tools `/beacon/mcp` serves AI clients over Streamable HTTP. Callers authenticate with an API key (`Authorization: Bearer sk-sem_…`) with the Execute scope, or with an Entra token mapped to a caller. - **Eight built-in tools**: `get_context`, `search`, `get_documentation`, `get_query_context`, `dry_run`, `query`, `ask` and `feedback`. - **Saved-query tools**: a published query appears as `q_` and runs only its approved version. - **Host endpoint tools**: a host can expose its own HTTP endpoints as `api_` tools. The caller's projects are resolved on every call, and every call is written to the audit log. See [MCP server](https://moberghr.github.io/beacon/mcp/server/), [Saved query tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/) and [Host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/). ## Grounding and knowledge base What Beacon gives an LLM to write SQL with: schema metadata, verified relationships, a business glossary, human-verified golden examples, learned patterns from usage, and project documentation. Curating it (verifying relationships, reviewing learned patterns, editing documentation) needs the Curate action in the project: ProjectAdmin members and Admins. See [Knowledge base & grounding](https://moberghr.github.io/beacon/mcp/knowledge-base/). ## Roles, project roles and functional permissions Three separate ideas: | Concept | Values | Decides | |---|---|---| | Global role | Admin, Editor, Viewer | The highest level a user can have in any project. Admin has everything. | | Project role | Viewer, Editor, ProjectAdmin | What a member may do in one project: read and run (Viewer), also write and query (Editor), also manage members and curate (ProjectAdmin). | | Functional permission | Approver, Auditor, SecurityAdmin, UserAdmin, DataSourceAdmin, MigrationOperator | One administrative duty split out of the Admin role. Grants no project data by itself. | API keys act as their owner, capped by their scope (Read or Execute), and never hold a functional permission. See [Authorization](https://moberghr.github.io/beacon/security/authorization/), [User management](https://moberghr.github.io/beacon/security/user-management/) and [API keys](https://moberghr.github.io/beacon/security/api-keys/). ## Deployment locks Some settings are fixed by the host's configuration and cannot be changed from the UI or API, not even by an Admin. Examples: `Beacon:Mcp:ForceReadOnly` and `Beacon:Mcp:ForcePiiDetection` (both on by default), `Beacon:Mcp:ForbidRowDataToLlm`, and `Beacon:Migrations:Enabled` (off by default). Only a configuration change lifts a lock. See [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/) and [Admin settings](https://moberghr.github.io/beacon/security/admin-settings/). ## Audit - **MCP audit log**: one row per MCP tool call, including refusals and errors, with every statement the call ran on a data source. Readable by Auditors and Admins. - **Setting history**: every admin setting, MCP setting and migration-write-target change, with old value, new value and who changed it. - **Security audit trail**: one append-only, signed event per security-relevant action (sign-ins, users and roles, API keys, project access, data sources, delivery, settings, approvals, ownership, curation, data quality, migrations and reads of audit data), by id only. Auditors read and verify it on the **Security Audit** page. - **Security log events**: structured log events (event ids 7101 to 7123) for membership changes, refused access, sign-in events and more. They go to the host's log pipeline. See [Security audit trail](https://moberghr.github.io/beacon/security/audit-trail/) and [MCP observability & audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/). --- # Quick start > Run the Beacon sample host from a clone and send your first alert end to end: data source, project, query, recipient and subscription. Source: https://moberghr.github.io/beacon/getting-started/quick-start/ This guide runs the sample host on your machine and walks you to a first alert: a scheduled query whose rows arrive at a webhook. It takes about 20 minutes. You will: 1. Start PostgreSQL and create Beacon's schema. 2. Run the sample host and the UI. 3. Create the first admin. 4. Add a data source, a project, a query, a recipient and a subscription. 5. Trigger a run and see the notification. ## Prerequisites | Tool | Version | Check | |---|---|---| | .NET SDK | 10 | `dotnet --version` | | Node.js and npm | Node 22, npm 11 | `node -v`, `npm -v` | | Docker | any recent | `docker --version` | | `dotnet-ef` | 10 | `dotnet ef --version` | | `openssl` | any | `openssl version` | - The UI's lockfile needs npm 11. If `npm -v` prints 10, run `npm install -g npm@11`. - Install the EF Core tools once with `dotnet tool install --global dotnet-ef`. - Instead of Docker you can use any PostgreSQL server that has the [pgvector](https://github.com/pgvector/pgvector) extension available. Clone the repository and run every command below from its root: ```bash git clone https://github.com/moberghr/beacon.git cd beacon ``` ## Step 1: Start PostgreSQL Start PostgreSQL with pgvector and a database named `beacon`: ```bash docker run -d --name beacon-pg \ -e POSTGRES_PASSWORD=beacon \ -e POSTGRES_DB=beacon \ -p 5432:5432 \ pgvector/pgvector:pg17 ``` This database holds Beacon's own data and the sample host's background jobs. In this guide it is also the database you monitor. ## Step 2: Set the two required secrets The sample host reads its connection string and encryption key from [user secrets](https://learn.microsoft.com/aspnet/core/security/app-secrets). User secrets are loaded only in the Development environment, which the launch profile in step 4 sets. ```bash dotnet user-secrets --project src/Beacon.SampleProject set \ "ConnectionStrings:BeaconContext" \ "Host=localhost;Port=5432;Database=beacon;Username=postgres;Password=beacon" dotnet user-secrets --project src/Beacon.SampleProject set \ "Beacon:EncryptionKey" "$(openssl rand -base64 32)" ``` This connection string, without TLS, is for the local container only; a deployed host's database connection belongs in its environment or secret store. The encryption key protects connection strings and other secrets at rest. Keep it: values encrypted with it cannot be read with another key. ## Step 3: Create Beacon's schema The sample host migrates only its job-runner tables at startup. Beacon's own schema is created with the EF Core tools: ```bash dotnet ef database update \ --project src/Beacon.Core.PostgreSql \ --startup-project src/Beacon.SampleProject \ --context PostgreSqlBeaconContext ``` The command builds the sample host and its dependencies, then applies every migration. The migrations create the `beacon` schema and the `vector` extension, so the login in the connection string needs the rights to create both (the `postgres` user in the container has them). The output ends with `Done.` ## Step 4: Run the sample host Trust the ASP.NET Core development certificate once, then start the host with the `https` launch profile: ```bash dotnet dev-certs https --trust dotnet run --project src/Beacon.SampleProject --launch-profile https ``` The profile sets `ASPNETCORE_ENVIRONMENT=Development` and listens on `https://localhost:7187` and `http://localhost:5296`. While no admin exists, the host prints a one-time setup token to the console (standard error, not the log): ```text Beacon first-run setup token (enter it on the setup page): 3f1c… ``` Copy it. A new token is generated every time the host starts until setup is done. ## Step 5: Open the UI Pick one way to serve the React UI. Either way, the UI talks to the host from step 4. ### Option A: Vite dev server (no build) In a second terminal: ```bash npm ci --prefix src/Beacon.UI/web npm run dev --prefix src/Beacon.UI/web ``` Open [http://localhost:5173/beacon/setup](http://localhost:5173/beacon/setup). Vite proxies `/beacon/api` and `/beacon/mcp` to `https://localhost:7187`. ### Option B: Build the UI once ```bash npm ci --prefix src/Beacon.UI/web npm run build --prefix src/Beacon.UI/web ``` The build writes to `src/Beacon.UI/wwwroot/beacon`. Restart the host from step 4 so it picks up the files, then open [https://localhost:7187/beacon/setup](https://localhost:7187/beacon/setup). :::tip[Exploring the UI without a backend] `npm run dev:mock --prefix src/Beacon.UI/web` serves the UI at `http://localhost:5173/beacon/` with mock data and no backend. Nothing you do there is saved. ::: ## Step 6: Create the first admin On the setup page, fill in: - **Setup token**: the token from step 4. - **Username**, and optionally **Email** and **Display name**. - **Password** and **Confirm password**: at least 8 characters, with an uppercase letter, a lowercase letter, a digit and a special character. Click **Create super admin**, then **Go to sign in**, and sign in with the account you just created. This first account is the super admin and holds the Admin role, so it can do everything in this guide. To confirm the host is healthy: ```bash curl -k https://localhost:7187/beacon/api/health # {"status":"ok"} ``` ## Step 7: Create a read-only login to monitor Beacon runs monitoring SQL read-only, but every data source should still use a login that cannot write. Create one in the same container: ```bash docker exec -i beacon-pg psql -U postgres -d beacon <<'SQL' CREATE ROLE beacon_reader LOGIN PASSWORD 'reader'; ALTER ROLE beacon_reader SET default_transaction_read_only = on; SQL ``` ## Step 8: Add a data source 1. Open **Data Sources** (`/beacon/data-sources`) and click **New data source**. 2. **Type**: choose **Database**, then **Next**. 3. **Connection**: - **Name**: `Local PostgreSQL` - **Database engine**: **PostgreSQL** - **Connection string**: `Host=localhost;Port=5432;Database=beacon;Username=beacon_reader;Password=reader` Click **Next**. 4. **Test & save**: click **Test connection** and wait for **Connected**, then click **Create data source**. Beacon encrypts the connection string and loads the database's metadata. Until the data source is linked into a project, only Admins can use it. :::note[Local development only] The local container has no TLS, so this connection does not authenticate the server. The sample host accepts it because its Development configuration sets `Beacon:DataSources:Transport:AllowInsecure`. Outside Development Beacon refuses it: a real data source needs a connection that verifies the server's certificate, for example `Host=db.example.com;Port=5432;Database=sales;Username=beacon_reader;Password=...;SSL Mode=VerifyFull`. See [Transport security (TLS)](https://moberghr.github.io/beacon/features/data-sources/#transport-security-tls). ::: ## Step 9: Create a project 1. Open **Projects** (`/beacon/projects`) and click **New project**. 2. **Project name**: `Getting started`. Add a **Description** if you like. 3. Under **Data sources**, pick `Local PostgreSQL`. 4. Click **Create project**. The project is restricted: only its members and Admins see it. You are its project admin. ## Step 10: Write a query An alert query returns rows when something needs attention, and no rows when all is well. A run with no rows never notifies. For a first test, use a query that always returns rows: the databases that have open connections. Beacon's own connections guarantee at least one. 1. Open **Queries** (`/beacon/queries`) and click **New query**. 2. **Query name**: `Databases with connections`. **Description**: `Connections per database`. 3. In the step: - **Step name**: `Connections` - **Target database**: `Local PostgreSQL` - **SQL query**: ```sql SELECT datname, numbackends FROM pg_stat_database WHERE numbackends > 0 ``` 4. Click **Save & run**. Beacon saves the query, opens it and runs it. You should see at least one row. For a real alert, raise the threshold in the `WHERE` clause so rows appear only when there is a problem, for example `WHERE numbackends > 50`. ## Step 11: Add a recipient A recipient must exist before you create the subscription. A webhook is the quickest to try: 1. Get a temporary public HTTPS endpoint from a request-inspection service such as [webhook.site](https://webhook.site). 2. Open **Recipients** (`/beacon/recipients`) and click **Add recipient**. 3. **Notification type**: **Webhook**. **Name**: `Test webhook`. **Webhook URL**: your endpoint's URL. 4. Click **Create recipient**. Other channels: | Channel | What you need | |---|---| | Email | The sample host sends email over SMTP. Set `Beacon:Smtp:Host`, `Beacon:Smtp:FromAddress`, `Beacon:Smtp:Username` and `Beacon:Smtp:Password` (and `Port`, default `465`; `EnableSsl`, default `true`) in user secrets and restart the host. | | Microsoft Teams | A Teams Workflows webhook URL. Beacon accepts Teams URLs on `*.webhook.office.com`, `*.logic.azure.com` and `*.environment.api.powerplatform.com`. | | Slack | A Slack incoming-webhook URL on `hooks.slack.com`. | | Jira | A connection string in the form `domain;project-key;email;api-key`. | See [Notifications](https://moberghr.github.io/beacon/features/notifications/) for the destination rules. ## Step 12: Create the subscription 1. Open **Subscriptions** (`/beacon/subscriptions`) and click **New subscription**. (The query's page also has **Add subscription**.) 2. **Query** step: - **Query**: `Databases with connections` - **Cron expression**: `*/5 * * * *` (every five minutes; standard 5-field cron) 3. **Notify** step: - **Send notification**: **Every execution** - **Recipients**: tick `Test webhook` 4. **Results** and **Anomaly**: keep the defaults. 5. **Review**: click **Create subscription**. The subscription runs as you, its owner. The sample host's job runner (Warp) picks it up on the schedule. ## Step 13: Trigger a run and check the result You do not need to wait five minutes: 1. On the subscription's page, click **Test now**. 2. Your webhook endpoint receives a `POST` with a JSON body: the subscription and data source names, the SQL, the row count and the rows. 3. Open **Notifications** (`/beacon/notifications`) to see the delivery, its status and the stored result. You have a working alert. Before you leave it running, click **Archive** on the subscription, or create a new one on a query with a real threshold. ## Troubleshooting | Symptom | Cause and fix | |---|---| | The host stops with `Beacon:DataProtection:ApplicationName must be set outside Development` | The host did not run in Development, so user secrets were not loaded either. Start it with `--launch-profile https`. | | The host stops with an `EncryptionKey` error | The key is missing or not valid base64 of 32 random bytes. Repeat step 2 and check `dotnet user-secrets --project src/Beacon.SampleProject list`. | | `More than one DbContext was found` from `dotnet ef` | Add `--context PostgreSqlBeaconContext`, as in step 3. | | `relation "…" does not exist` in the host's log | Beacon's schema was not migrated. Run step 3. | | `extension "vector" is not available` | The PostgreSQL server lacks pgvector. Use the `pgvector/pgvector` image or install the extension on your server. | | `404` at `/beacon/setup` | The UI is not built. Use Vite (option A) or build it (option B) and restart the host. | | Nothing at `https://localhost:7187/` | The UI lives at `/beacon`, not at the site root. | | The browser warns about the certificate | Run `dotnet dev-certs https --trust` and restart the browser. | | The subscription runs but nothing arrives | The query returned no rows (no notification by design), the trigger skipped the run, or the destination is not allowed. Check the run's status in **Notifications**. | ## Next steps - [Core concepts](https://moberghr.github.io/beacon/getting-started/concepts/): how projects, queries and subscriptions fit together. - [Run the sample host](https://moberghr.github.io/beacon/getting-started/installation/): everything else the sample host can do. - [Queries](https://moberghr.github.io/beacon/features/queries/): multi-step queries and parameters. - [MCP server](https://moberghr.github.io/beacon/mcp/server/): connect an AI assistant to the same data. - [Integrate Beacon](https://moberghr.github.io/beacon/integrate/): run Beacon inside your own application. --- # Run the sample host > Reference for Beacon.SampleProject, the sample host in the repository: what it wires, the configuration it reads, ports, Warp jobs, the UI build, running outside Development and publishing a release. Source: https://moberghr.github.io/beacon/getting-started/installation/ `src/Beacon.SampleProject` is the composition root in the Beacon repository: an ASP.NET Core application that wires every Beacon package together. It is how you evaluate Beacon and how contributors run it. The [Quick start](https://moberghr.github.io/beacon/getting-started/quick-start/) takes you from a clone to a first alert; this page is the reference for everything else the sample does. :::caution[A reference, not a product] The sample host is never shipped. It hard-codes choices a real deployment makes for itself: Warp as the job runner, SMTP for email, Beacon's own user store, no Data Protection key encryption. Run Beacon in production by embedding the packages in your own host. See [Integrate Beacon](https://moberghr.github.io/beacon/integrate/). ::: ## What the sample includes | Area | What the sample wires | Where | |---|---|---| | Beacon services | The host chain (`AddBeacon(...)`, `.AddAI()`, `.AddMcp()`, `.AddApi()`, `.AddCookieAuth()`, `.AddOidc()`, then `UseBeaconPipeline()` and `MapBeacon()`) with AI on, authorization on, the login form, Beacon's user store (`DatabaseAuthenticationProvider`) and user management with first-run setup. The approval workflow is not enabled. | `Program.cs` | | Metadata database | PostgreSQL (`.UsePostgreSql()`), connection string `ConnectionStrings:BeaconContext`. | `Program.cs` | | Connectors | All nine: PostgreSQL, SQL Server, MySQL, CloudWatch, Azure Synapse, Snowflake, Databricks, BigQuery and REST API. | `Program.cs` | | Background jobs | [Moberg Warp](https://moberghr.github.io/warp/) on PostgreSQL, with `BeaconScheduler` as the `IBeaconScheduler` implementation and one Warp job per Beacon operation. Dashboard at `/warp`. | `Warp/`, `Services/BeaconScheduler.cs` | | Sign-in | Cookie sign-in under `/beacon`, Data Protection keys in Beacon's database, optional OIDC/SSO, and bearer JWT for MCP clients when OIDC has a `McpJwksEndpoint`. | `Program.cs` | | API, UI and MCP | REST API and SignalR hub under `/beacon/api`, the React UI at `/beacon`, the MCP server at `/beacon/mcp`, the MCP discovery documents, and the OpenAPI document at `/openapi/v1.json`. | `Program.cs` | | Email | `BeaconMailSender`, an SMTP adapter configured with `Beacon:Smtp:*`. | `Services/BeaconMailSender.cs` | | Error responses | Exceptions under `/beacon/api` become RFC 7807 `application/problem+json` responses (`UseBeaconApiExceptionHandler`, added by `UseBeaconPipeline`). | `Program.cs` | | Telemetry | OpenTelemetry traces and metrics, only when `OTEL_EXPORTER_OTLP_ENDPOINT` is set. | `Program.cs` | | Reverse proxy | Forwarded headers from trusted proxies, only when `Beacon:ForwardedHeaders:Enabled` is `true`. | `Program.cs` | | Timeouts | Kestrel keep-alive and request-header timeouts and the default `HttpClient` timeout set to 5 minutes, for long AI calls (`builder.ConfigureBeaconTimeouts()`). | `Program.cs` | The Warp jobs, `BeaconMailSender`, the telemetry and forwarded-headers blocks are sample code, not part of any package. A host that embeds Beacon provides its own. The error responses, the timeouts and the OIDC bearer JWTs come from helpers in `Moberg.Beacon.Api`, and the chain's `UseBeaconPipeline` and `MapBeacon` from `Moberg.Beacon.Gateway`. ## Configuration The sample reads standard ASP.NET Core configuration: `appsettings.json`, `appsettings.{Environment}.json`, user secrets (Development only), environment variables and command-line arguments. Environment variables use `__` for `:`, for example `Beacon__EncryptionKey`. `appsettings.json` holds the non-secret defaults and commented examples. `appsettings.Development.json` sets only logging. Secrets never go in either file. | Key | Required | What it does | |---|---|---| | `ConnectionStrings:BeaconContext` | Yes | Beacon's metadata database (PostgreSQL in the sample). | | `Beacon:EncryptionKey` | Yes | Base64 of at least 32 random bytes. Encrypts connection strings and other secrets at rest. Checked at startup in every environment. | | `ConnectionStrings:WarpContext` | No | PostgreSQL database for Warp. Defaults to `BeaconContext`. | | `Beacon:BaseUrl` | No | Base for links in notifications, built as `{BaseUrl}/notifications/{id}`. Include `/beacon`, for example `https://beacon.example.com/beacon`. Can be changed later in **Admin Settings**. | | `Beacon:PublicBaseUrl` | Outside local use | Public origin advertised in the MCP discovery documents. Unset, Beacon uses the request's Host header and logs a warning. | | `Beacon:DataProtection:ApplicationName` | Outside Development | Data Protection application name, one per deployment. | | `Beacon:DataProtection:AllowUnencryptedKeys` | No | Acknowledges a key ring stored without encryption at rest. See [Running outside Development](#running-outside-development). | | `Beacon:UserManagement:SetupToken` | Recommended outside Development | First-run setup token, at least 32 characters. Unset, each process prints its own random token to the console. | | `Beacon:UserManagement:LocalLogin:Enabled`, `BreakGlassUserNames` | No | Password sign-in on or off, and up to three break-glass accounts that keep a password when it is off. | | `Beacon:Authentication:Oidc:*` | No | OIDC/SSO. See [Sign-in with OIDC](#sign-in-with-oidc-and-jwt-for-mcp). | | `Beacon:Mcp:Callers:*` | No | Entra callers admitted to `/beacon/mcp`. See [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/). | | `Beacon:ForwardedHeaders:Enabled`, `KnownProxies` | No | Trust `X-Forwarded-For` and `X-Forwarded-Proto` from the listed proxy IP addresses. An empty list trusts none. | | `Beacon:Smtp:Host`, `FromAddress`, `Username`, `Password`, `Port`, `EnableSsl` | For email | SMTP for email recipients. `Port` defaults to `465`, `EnableSsl` to `true`. A missing required key fails the send, not the startup. | | `Beacon:LLM:*` | For AI features | The LLM provider. See [AI providers](https://moberghr.github.io/beacon/integrate/ai/). | | `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_SERVICE_NAME` | No | Turn on OpenTelemetry export. The service name defaults to `beacon`. | Every other key (query limits, MCP locks, notification policy, API keys, sessions, authorization) is a Beacon library setting and works the same in any host. See [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/). ## Ports and launch profiles `src/Beacon.SampleProject/Properties/launchSettings.json` has three profiles. All of them set `ASPNETCORE_ENVIRONMENT=Development`. | Profile | URLs | Use | |---|---|---| | `http` | `http://localhost:5296` | The default for `dotnet run` without `--launch-profile`. | | `https` | `https://localhost:7187`, `http://localhost:5296` | The one to use. The Vite dev server proxies to `https://localhost:7187`. | | `IIS Express` | IIS Express ports | Visual Studio on Windows. | ```bash dotnet run --project src/Beacon.SampleProject --launch-profile https ``` With `--no-launch-profile` the environment is Production: user secrets and `appsettings.Development.json` are not loaded, and the host refuses to start until the [production settings](#running-outside-development) are in place. In Development, HTTP requests are redirected to HTTPS except under `/beacon/mcp`, so local MCP clients can connect over `http://localhost:5296/beacon/mcp`. Outside Development every path is redirected, including `/beacon/mcp`. ## Beacon's database The sample host does not create or migrate Beacon's schema. Run the EF Core migrations yourself, before the first start and after every update: ```bash dotnet tool install --global dotnet-ef dotnet ef database update \ --project src/Beacon.Core.PostgreSql \ --startup-project src/Beacon.SampleProject \ --context PostgreSqlBeaconContext ``` The command reads the same configuration as the host, in the Development environment, so user secrets apply. The migrations create the `beacon` schema and the pgvector `vector` extension, so the login needs the rights for both, or a DBA creates them first. The sample registers PostgreSQL for Beacon's data. To use SQL Server instead, change `.UsePostgreSql()` to `.UseSqlServer()` in `Program.cs`, apply the migrations from `src/Beacon.Core.SqlServer`, and set `ConnectionStrings:WarpContext` to a PostgreSQL database, because Warp needs PostgreSQL. See [Metadata database](https://moberghr.github.io/beacon/integrate/database/). ## Background jobs (Warp) At startup the sample migrates Warp's own `warp` schema and registers its recurring jobs. Subscriptions are scheduled through `BeaconScheduler` as you create them. | Job | Schedule | What it does | |---|---|---| | `mcp-learning-aggregate` | `0 */6 * * *` | Mines MCP usage signals into learned patterns. | | `mcp-learning-cleanup` | `0 3 * * *` | Deletes learning signals past their retention. | | `mcp-audit-retention` | `30 3 * * *` | Purges MCP audit rows older than `Beacon:Mcp:Audit:RetentionDays`. Does nothing when that is unset. | | `mcp-embedding-reindex` | `0 */12 * * *` | Re-indexes metadata and examples for semantic search. | | `mcp-docchunk-reindex` | `0 */12 * * *` | Re-chunks and re-embeds project documentation. | Every start also enqueues one run of the re-encryption job, which moves stored values still under a key in `Beacon:PreviousEncryptionKeys`, or still in the legacy encryption format, to the current key and format. A run with nothing to move writes nothing. - Warp polls every second and runs at most `min(2 × CPU count, 20)` workers. - Failed jobs are not retried automatically. They stay failed for you to investigate. - The dashboard at `/warp` is for signed-in users with the Admin role. It sits outside `/beacon`, so the sample lists `/warp` in `Beacon:Authentication:AdditionalPaths` (`appsettings.json`): Beacon's session authenticates `/beacon` and the listed paths only. See [Where Beacon credentials apply](https://moberghr.github.io/beacon/getting-started/configuration/#where-beacon-credentials-apply). See [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/) for implementing `IBeaconScheduler` on another job runner. ## The UI The React app lives in `src/Beacon.UI/web`. The `Beacon.UI` project serves the built files from `src/Beacon.UI/wwwroot/beacon` at `/beacon`. The repository does not contain a built UI. | Goal | Command (from the repository root) | |---|---| | Install dependencies | `npm ci --prefix src/Beacon.UI/web` | | Build once for the host | `npm run build --prefix src/Beacon.UI/web`, then restart the host. Output: `src/Beacon.UI/wwwroot/beacon`. | | Develop with hot reload | `npm run dev --prefix src/Beacon.UI/web`, then open `http://localhost:5173/beacon/`. Proxies `/beacon/api` and `/beacon/mcp` to `https://localhost:7187`, or to `BEACON_BACKEND_URL` when set. | | Explore without a backend | `npm run dev:mock --prefix src/Beacon.UI/web`, mock data in the browser. | | Run the UI tests | `npm test --prefix src/Beacon.UI/web` | | Regenerate the API client | `npm run codegen --prefix src/Beacon.UI/web`, with the host running. Reads `/openapi/v1.json`. | A Debug build of the .NET solution does not build the React app. A Release build does, and so does a Debug build with `-p:BuildReactInDebug=true`. Both run `npm ci` first when `node_modules` is missing. The UI needs Node.js 22 and npm 11. ## Sign-in with OIDC and JWT for MCP The sample always offers Beacon's login form and user store. To add single sign-on, fill in `Beacon:Authentication:Oidc` and set `Enabled` to `true`: ```json { "Beacon": { "Authentication": { "Oidc": { "Enabled": true, "Authority": "https://login.microsoftonline.com//v2.0", "ClientId": "", "AllowedTenants": [""], "DisplayName": "Microsoft", "McpJwksEndpoint": "https://login.microsoftonline.com//discovery/v2.0/keys" } } } } ``` - Put `ClientSecret` in user secrets or an environment variable (`Beacon__Authentication__Oidc__ClientSecret`), never in the file. - Startup fails with SSO on and no `AllowedTenants`, unless `AllowAnyTenant` is `true` for a provider that issues no tenant claim. - `McpJwksEndpoint`: when it is set, a host that calls `.AddOidc()` (or `AddBeaconOidcMcpJwtAuthentication`), as the sample does, also accepts bearer JWTs on the API and on `/beacon/mcp`, validated against those signing keys with the `Authority` as issuer and the `ClientId` as audience. Which tokens become MCP callers is decided by `Beacon:Mcp:Callers`. Every OIDC option is in [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/). How sign-in works is in [Sign-in & identity](https://moberghr.github.io/beacon/integrate/authentication/). ## Running outside Development Outside Development (`ASPNETCORE_ENVIRONMENT` set to anything but `Development`, or unset), the sample needs more configuration before it starts: 1. **Provide secrets through the environment or a secret store.** User secrets are not loaded. Set at least `ConnectionStrings__BeaconContext` and `Beacon__EncryptionKey`. 2. **Set `Beacon:DataProtection:ApplicationName`.** One name per deployment, for example `beacon-staging`. Without it the host does not start. 3. **Encrypt the Data Protection keys at rest.** Outside Development the host refuses to start while the key ring, which protects sign-in cookies, is stored unencrypted. The sample chains no `ProtectKeysWith*` call: add one to the builder `AddBeaconDataProtection` returns in `Program.cs` (for example `ProtectKeysWithAzureKeyVault` or `ProtectKeysWithCertificate`), or, if the database is protected by other means, set `Beacon:DataProtection:AllowUnencryptedKeys` to `true` and accept the startup warning. 4. **Set `Beacon:UserManagement:SetupToken`** to a secret of at least 32 characters, so the first admin can be created with a token you control, and every replica accepts the same one. 5. **Set `Beacon:PublicBaseUrl` and `Beacon:BaseUrl`** to the public origin and the public UI address (with `/beacon`). 6. **Configure the listening address**, for example `ASPNETCORE_URLS`. Without a launch profile ASP.NET Core does not use the ports above. 7. **Behind a reverse proxy**, set `Beacon:ForwardedHeaders:Enabled` and list the proxy addresses in `KnownProxies`, so the login rate limiter and cookie security see the client's address and scheme. The [Production checklist](https://moberghr.github.io/beacon/integrate/production/) covers the rest for a real deployment. ## Building a release ```bash dotnet publish src/Beacon.SampleProject -c Release -o ./publish ``` The Release build runs `npm ci` (when needed) and `npm run build` for the UI, so the output includes it under `wwwroot/beacon`. Run it with: ```bash cd publish ASPNETCORE_ENVIRONMENT=Production dotnet Beacon.SampleProject.dll ``` with the settings from [Running outside Development](#running-outside-development) supplied through the environment. ## URLs | URL | What | |---|---| | `/beacon` | The React UI. The site root `/` belongs to the host. | | `/beacon/setup` | First-run setup, while no admin exists. | | `/beacon/login` | Sign-in. | | `/beacon/api/health` | Health check: `{"status":"ok"}` when the metadata database answers. | | `/beacon/api/*` | REST API and the SignalR hub (`/beacon/api/hub`). | | `/beacon/mcp` | MCP server. | | `/openapi/v1.json` | OpenAPI document. | | `/warp` | Warp dashboard (Admin only; listed in `Beacon:Authentication:AdditionalPaths`). | The full list is in [Routes & endpoints](https://moberghr.github.io/beacon/reference/routes/). ## Troubleshooting | Symptom | Cause and fix | |---|---| | `Beacon:DataProtection:ApplicationName must be set outside Development` | The host runs outside Development. Use `--launch-profile https` locally, or follow [Running outside Development](#running-outside-development). | | Startup refuses Data Protection keys "persisted without encryption at rest" | Outside Development without key encryption. See step 3 of [Running outside Development](#running-outside-development). | | `Beacon:EncryptionKey` errors | The key is missing, too short, printable text or a published sample value. Generate one with `openssl rand -base64 32`. | | `relation "…" does not exist` | Beacon's schema is not migrated. See [Beacon's database](#beacons-database). | | `extension "vector" is not available` | Install pgvector on the PostgreSQL server, or use the `pgvector/pgvector` image. | | `404` under `/beacon` | The UI is not built. See [The UI](#the-ui). | | Subscriptions never run | Warp is not running or cannot reach PostgreSQL. Check `/warp` and the host log. | More in [Troubleshooting](https://moberghr.github.io/beacon/integrate/troubleshooting/). ## Embedding Beacon in your own application This page covers the sample only. To add Beacon to your own ASP.NET Core application (packages, `Program.cs`, middleware order, migrations at startup, your job runner and your sign-in), follow [Integrate Beacon](https://moberghr.github.io/beacon/integrate/). --- # Cookbooks > Task-shaped Beacon recipes, each one start to finish with a check that it works and a prompt you can hand to your coding agent: connect AI clients, publish tools, embed Beacon, single sign-on, alerting, data contracts and audit. Source: https://moberghr.github.io/beacon/cookbooks/ Each recipe takes one goal from nothing to a verified result: what you need, the steps, how to check it works, what to do when it does not, and where to go next. Each one opens with a prompt for your AI, because every recipe is also served as Markdown that an agent can follow. :::tip[Hand any recipe to your AI] ```text wrap Read https://moberghr.github.io/beacon/llms-cookbooks.txt, pick the recipe for: , tell me which one you picked, then follow it step by step and show me each "Check it works" result. Ask me before every decision a recipe marks as one. ``` Or point the agent at one recipe: every page is at its URL plus `.md`, for example `https://moberghr.github.io/beacon/cookbooks/connect-coding-agent.md`. ::: ## Start here | If you want to | Recipe | Time | |---|---|---| | Let Claude act in **your own application** as your users, through your endpoints | [MCP gateway for your application](https://moberghr.github.io/beacon/mcp/gateway/) | about 1 hour, plus the Entra registration | | Point **Claude Code, Cursor or VS Code** at a running Beacon | [Connect a coding agent](https://moberghr.github.io/beacon/cookbooks/connect-coding-agent/) | about 10 minutes | | Run **Beacon inside your application**, with its UI, monitoring and MCP server | [Embed Beacon (PostgreSQL + Warp)](https://moberghr.github.io/beacon/cookbooks/embed-postgresql-warp/) | about 45 minutes | | Get **good answers** from an agent connected to Beacon | [Ask your data with an agent](https://moberghr.github.io/beacon/cookbooks/ask-your-data/) | about 15 minutes | ## Connect AI to your data | Recipe | You end up with | Time | |---|---|---| | [Connect Claude Code, Cursor or VS Code](https://moberghr.github.io/beacon/cookbooks/connect-coding-agent/) | A coding agent calling Beacon's MCP tools with a project-scoped API key kept out of your files | 10 min | | [Add Beacon to claude.ai and Claude Desktop with Entra sign-in](https://moberghr.github.io/beacon/cookbooks/claude-connector-entra/) | A Claude custom connector whose users sign in with Microsoft Entra and reach their projects | 45 min | | [Ask your data questions with an AI agent](https://moberghr.github.io/beacon/cookbooks/ask-your-data/) | An agent that orients, grounds, validates its SQL with `dry_run`, and rates answers | 15 min | | [Publish an approved saved query as an MCP tool](https://moberghr.github.io/beacon/cookbooks/saved-query-tool/) | A reviewed, parameterised `q_` tool instead of free SQL | 20 min | | [Give an AI agent access to one project only](https://moberghr.github.io/beacon/cookbooks/agent-one-project/) | An agent confined to one restricted project, with masking and caps, and its calls in the audit | 20 min | ## Make your app AI-ready | Recipe | You end up with | Time | |---|---|---| | [MCP gateway for your application](https://moberghr.github.io/beacon/mcp/gateway/) | Your application as an MCP server: Entra sign-in, your endpoints as tools, your docs as grounding | 1 h | | [Expose an endpoint as an MCP tool, safely, and test it](https://moberghr.github.io/beacon/cookbooks/endpoint-as-tool/) | A self-service read tool and a confirmed write tool, with an end-to-end test | 30 min | | [Ground AI answers in your own docs and data model](https://moberghr.github.io/beacon/cookbooks/ground-with-your-docs/) | Your Markdown docs and EF Core model in the answers agents give | 25 min | ## Embed Beacon | Recipe | You end up with | Time | |---|---|---| | [Embed Beacon in an existing ASP.NET Core app (PostgreSQL + Warp)](https://moberghr.github.io/beacon/cookbooks/embed-postgresql-warp/) | Beacon's UI at `/beacon`, its API and MCP server, on your sign-in and job runner | 45 min | | [Embed Beacon on SQL Server with Hangfire](https://moberghr.github.io/beacon/cookbooks/embed-sqlserver-hangfire/) | The same on SQL Server, with your own Hangfire scheduler | 40 min | | [Sign in to Beacon with Microsoft Entra ID](https://moberghr.github.io/beacon/cookbooks/entra-sso/) | Single sign-on for Beacon's UI, with the first admin and a break-glass account | 30 min | ## Monitor & govern | Recipe | You end up with | Time | |---|---|---| | [Alert Slack or Teams when a nightly check fails](https://moberghr.github.io/beacon/cookbooks/alert-on-failed-check/) | A scheduled check that posts to a channel and opens a task when it finds rows | 20 min | | [Score a table with a data contract](https://moberghr.github.io/beacon/cookbooks/data-contract/) | A contract of quality rules, a score on a schedule, and an alert when it drops | 20 min | | [Ship the MCP audit and security events to your SIEM](https://moberghr.github.io/beacon/cookbooks/audit-to-siem/) | Every agent call and security event in your log pipeline, with traces and metrics | 30 min | ## Prompts The [prompt library](https://moberghr.github.io/beacon/cookbooks/prompts/) has ready-made prompts for coding agents in your repository, for asking your data questions through Beacon, for people using a gateway application's tools, and for admins. ## Missing a recipe? Open an [issue](https://github.com/moberghr/beacon/issues) with the goal you had and what you tried. For anything not covered yet, `llms-full.txt` gives an agent the whole documentation in one file. --- # Prompt library > Copy-paste prompts for working with Beacon through AI: for coding agents wiring Beacon into a repository, for people asking their data questions through Beacon's MCP tools, for people using a gateway application's tools, and for admins running Beacon. Source: https://moberghr.github.io/beacon/cookbooks/prompts/ Prompts that work, grouped by who types them and where. Replace the parts in ``. Each prompt names the Beacon page the AI should read, so the answer follows the current docs and not the model's memory of them. :::tip[The pattern behind every prompt] 1. **Point at the source:** a `…/llms-*.txt` bundle or a page's `.md` URL. 2. **Say the goal** in one sentence. 3. **Keep the decisions yours:** "ask me before every decision the guide marks as one". 4. **Ask for proof:** "show me each Check it works result", "show me the SQL before you run it". ::: ## In your repository (coding agents) For Claude Code, Cursor, Copilot or any agent that edits code. With Claude Code, the [Beacon plugin](https://moberghr.github.io/beacon/getting-started/with-ai/#the-claude-code-plugin) adds skills that do the same with less typing. ### Set Beacon up ```text wrap title="MCP gateway for this application" Add an MCP gateway to this ASP.NET Core app with Beacon. Read https://moberghr.github.io/beacon/llms-gateway.txt first and follow it step by step. Before writing code, read the app and tell me: how users sign in, where the users table and its Entra object id are, which database and migrator we use, and which endpoints look like self-service reads. Then ask me the decisions the guide lists. ``` ```text wrap title="Embed Beacon's data product" Integrate Beacon into this ASP.NET Core app, with for Beacon's own data and for jobs. Read https://moberghr.github.io/beacon/llms-integrate.txt and follow its checklist in order. Ask me before every decision it marks as one, keep secrets in user secrets, and show me each verification step. ``` ```text wrap title="Single sign-on with Microsoft Entra ID" Turn on Entra ID single sign-on for Beacon in this app. Read https://moberghr.github.io/beacon/cookbooks/entra-sso.md and follow it. Print the app registration steps for me to run in the portal instead of changing the tenant yourself. ``` ```text wrap title="Upgrade to Beacon 5.0" Upgrade this application from Beacon 4.x to 5.0. Read https://moberghr.github.io/beacon/getting-started/whats-new.md and apply every item under "Upgrade from 4.x" that applies. List the ones that did not apply and why. Build and run the tests afterwards. ``` ### Tools for agents ```text wrap title="Expose one endpoint as a tool" Make available to Claude as a Beacon MCP tool. Read https://moberghr.github.io/beacon/cookbooks/endpoint-as-tool.md first. Check the endpoint against its tool review checklist before marking it, bind the person's id from the caller, write an agent-facing description field by field, and add a Moberg.Beacon.Testing test that proves it refuses somebody else's data. ``` ```text wrap title="Review the tools we already expose" List every endpoint in this repository marked with WithBeaconTool or [BeaconTool]. For each, check it against the tool review checklist in https://moberghr.github.io/beacon/cookbooks/endpoint-as-tool.md and report: self-service or not, what data it returns, whether ReadOnly is declared, whether the person's id is bound from the caller, and whether a test pins it. Do not change anything yet. ``` ```text wrap title="Publish a saved query as a tool" Walk me through publishing our approved saved query "" as an MCP tool. Follow https://moberghr.github.io/beacon/cookbooks/saved-query-tool.md and tell me who has to do which step. ``` ### Fix and harden ```text wrap title="A startup error" Beacon refused to start with this message: . Read https://moberghr.github.io/beacon/integrate/troubleshooting.md and the page the message points to, explain the cause in two sentences, then fix it in this repository. ``` ```text wrap title="Claude cannot connect to our MCP endpoint" Claude cannot connect to https:///beacon/mcp. The error is: . Read https://moberghr.github.io/beacon/mcp/gateway.md (sections 6 to 8) and tell me what to check, in order. Then run: beacon-doctor mcp https:///beacon/mcp and explain every WARN and FAIL it prints. ``` ```text wrap title="Production review" Review this application's Beacon set-up against https://moberghr.github.io/beacon/integrate/production.md. Go through every checklist item and answer pass, fail or not applicable, with the file and line that shows it. Do not change anything; give me the list. ``` ## Ask your data For Claude, Claude Desktop, Cursor or any MCP client connected to Beacon's MCP server. The recipe [Ask your data questions with an AI agent](https://moberghr.github.io/beacon/cookbooks/ask-your-data/) explains the workflow behind these. ```text wrap title="Start a session" Use Beacon's get_context to describe my project: its data sources, the main tables and how well they are documented. Do not run any queries yet. ``` ```text wrap title="Find where something lives" Where does Beacon keep ? Search for it, read the documentation of the most likely tables, and tell me which columns hold the amount, the date and the customer. ``` ```text wrap title="Write the SQL, check it, then run it" I need . Get the grounding context for that question, write the SQL yourself, validate it with dry_run and show me the SQL and the verdict. Run it with query only after I say go. ``` ```text wrap title="A quick answer" Ask Beacon: "" Show me the SQL it generated with the answer. ``` ```text wrap title="Explain a number" Explain where the figure <12,430 active customers> comes from: the SQL, the tables it reads, and the filter that decides who counts. If Beacon has no definition of the term, say so instead of guessing. ``` ```text wrap title="Compare two periods" Compare in and by . Use one query, validate it with dry_run first, and point out the three biggest changes. Say if any result was truncated or masked. ``` ```text wrap title="Use the reviewed reports first" Which saved-query tools (q_…) do I have? If one of them answers "", run it instead of writing SQL, and tell me which one you used. ``` ```text wrap title="Rate the answer" That answer was . Record it with Beacon's feedback tool and the signal_id from the answer<, with this corrected SQL: …>. ``` ```text wrap title="A brief for your agent's instructions" Call get_context with format "agents_md" and turn the result into a short section I can paste into AGENTS.md: the data sources, the main tables and what they mean, and the rules for using Beacon's tools. ``` ## In an application with a Beacon gateway For people whose own application has a Beacon MCP gateway: Claude acts as you, through the application's own endpoints, and sees only what the application lets you see. ```text wrap title="What can you do here?" List the tools you have and, for each, one line on what it does and whether it changes anything. ``` ```text wrap title="A read" Show me my in , newest first, with their status. ``` ```text wrap title="A change (the client asks before it runs)" Request in . Before you call the tool, tell me exactly what you will send. ``` ## Running Beacon (admins) ```text wrap title="Design project boundaries" We have these teams and databases: . Read https://moberghr.github.io/beacon/security.md and propose Beacon projects, which data sources each links, who is a member with which role, and which projects stay restricted. Explain each choice in one line. ``` ```text wrap title="A key for one agent" I want an AI agent to read only the project. Follow https://moberghr.github.io/beacon/cookbooks/agent-one-project.md and give me the exact steps for our Beacon, the key settings and the project settings I should check. ``` ```text wrap title="Read the MCP audit" Here is an export of Beacon's MCP audit: . Read https://moberghr.github.io/beacon/mcp/observability-and-audit.md for what the fields mean, then summarise: which callers ran what, which calls were refused and why, and anything that looks unusual. ``` ```text wrap title="Lock it down" Read https://moberghr.github.io/beacon/mcp/server.md (Deployment locks and ceilings) and propose the Beacon:Mcp configuration for a regulated production deployment, with one line per setting on why. ``` ## Writing your own - **Name the page.** "Read `https://moberghr.github.io/beacon/.md`" is short, and the agent works from current docs. - **Ask for the plan first** on anything that changes code, a tenant or a deployment: "tell me what you will do, then wait". - **Name the tool when it matters.** "Validate it with `dry_run`" is more reliable than "be careful". - **Ask for evidence.** The SQL, the row count, the `beacon-doctor` output, the test that pins it. - **Never paste secrets.** Give environment variable names, not API keys or client secrets. --- # Connect Claude Code, Cursor or VS Code > Connect a coding agent to Beacon's MCP server with a project-limited API key kept out of your repository, and prove the connection with a first tool call. Source: https://moberghr.github.io/beacon/cookbooks/connect-coding-agent/ | | | |---|---| | **You end up with** | Claude Code, Cursor or VS Code calling Beacon's tools (`get_context`, `search`, `query` and the rest) against one project, with the key outside every committed file | | **Time** | about 10 minutes | | **You need** | A running Beacon with MCP mapped (`app.MapBeaconMcp()`, see [MCP and AI clients](https://moberghr.github.io/beacon/mcp/connect/#wiring)), a Beacon account with the Editor role or above, and a project with at least one data source | | **Applies to** | Beacon 5.0 · Full Beacon | :::tip[Let your AI do it] Paste this into your coding agent. It cannot create the key for you (keys are created in a signed-in browser session only), so it will stop and ask you for it. ```text wrap Read https://moberghr.github.io/beacon/cookbooks/connect-coding-agent.md and follow it step by step in this repository. Ask me for the Beacon URL and which client I use. I create the API key myself in the Beacon UI and put it in the BEACON_MCP_API_KEY environment variable: never ask me to paste the key into the chat, never write it into a file that is committed, and never commit .cursor/mcp.json or .vscode/mcp.json with a key in it. Show me the result of each "Check it works" item. ``` ::: ## Before you start | Decision | Options | Default | |---|---|---| | Client | Claude Code, Cursor, VS Code | The one you use; the steps are independent | | Where the config lives | The repository (shared with the team, no key inside) or your user profile (yours alone) | The repository, with the key read from an environment variable | | Projects the key reaches | One project, or several | One. With several, the agent must pass `project_id` on every call | | Expiry | A date within `Beacon:ApiKeys:MaxLifetimeDays` (365 by default) | 90 days | ## 1. Create an API key 1. In Beacon, open **API Keys** (`/beacon/api-keys`) and choose **Generate key**. 2. **Key name**: something that says where it is used, for example `Cursor on my laptop`. 3. **Scopes**: tick **Execute**. The MCP endpoint requires it; a `Read` key gets `403`. The checkbox is disabled unless you hold the Editor role or above. 4. **Projects**: pick the project the agent should use. At least one is required. 5. **Expiration date**: set a short one, or leave it empty for 90 days. 6. Generate the key and copy it (`sk-sem_…`). It is shown once; Beacon keeps only a hash. The key acts as you, in the projects you pick, and never with more than your own role allows. If you later lose the Editor role, the key acts as a Read key and Beacon's MCP endpoint refuses it (403) until you regain the role. More on this in [API keys](https://moberghr.github.io/beacon/security/api-keys/#scopes). ## 2. Put the key in an environment variable Set it in your shell profile or secret manager, not in the repository: ```bash export BEACON_MCP_API_KEY='sk-sem_…' ``` Start the client from a shell where the variable is set (an IDE launched from the dock does not see variables set only in a terminal session). ## 3. Add the server to your client Use `https://beacon.example.com/beacon/mcp` below; replace it with your Beacon's public URL. ### Claude Code Shared with the team: commit this `.mcp.json` at the repository root. Claude Code expands `${BEACON_MCP_API_KEY}` when it starts, so no key is in the file. ```json { "mcpServers": { "beacon": { "type": "http", "url": "https://beacon.example.com/beacon/mcp", "headers": { "Authorization": "Bearer ${BEACON_MCP_API_KEY}" } } } } ``` Only for you: add it from the command line instead. ```bash claude mcp add --transport http beacon https://beacon.example.com/beacon/mcp \ --header "Authorization: Bearer $BEACON_MCP_API_KEY" ``` Your shell expands `$BEACON_MCP_API_KEY` before Claude Code sees it, so the key is stored in Claude Code's own configuration under your user profile, not in the repository. Do not run this command in a directory whose configuration you commit. ### Cursor Put the file in your user profile, `~/.cursor/mcp.json`, so it is never in a repository: ```json { "mcpServers": { "beacon": { "url": "https://beacon.example.com/beacon/mcp", "headers": { "Authorization": "Bearer ${env:BEACON_MCP_API_KEY}" } } } } ``` If you want the project-level `.cursor/mcp.json` instead, add it to `.gitignore` or use the `${env:…}` form above. Check Cursor's own documentation if the `${env:…}` interpolation does not resolve in your version; the header and URL are all Beacon needs. ### VS Code `.vscode/mcp.json` prompts for the key once and keeps it out of the file: ```json { "inputs": [ { "type": "promptString", "id": "beacon-key", "description": "Beacon API key", "password": true } ], "servers": { "beacon": { "type": "http", "url": "https://beacon.example.com/beacon/mcp", "headers": { "Authorization": "Bearer ${input:beacon-key}" } } } } ``` ## 4. Keep the key out of git - [ ] `git grep sk-sem_` finds nothing in the repository. - [ ] `.cursor/mcp.json` is either absent or contains `${env:…}` only; `.vscode/mcp.json` contains `${input:…}` only. - [ ] If a key ever reached a commit, revoke it on **API Keys** (**Revoke**, effective on the next request) and create a new one. ## Check it works **From the client.** Restart it, then ask: *"Use Beacon's get_context to describe my project."* The answer lists the project's data sources and tables. In Claude Code, `/mcp` shows the `beacon` server as connected. - [ ] The client lists Beacon's built-in tools (all eight unless the host chose fewer): `get_context`, `search`, `get_documentation`, `get_query_context`, `ask`, `query`, `dry_run`, `feedback`. - [ ] `get_context` answers with your project. **From a terminal**, independent of any client. Beacon's discovery card needs no key: ```bash BEACON=https://beacon.example.com curl -s $BEACON/.well-known/mcp/server-card.json | jq '.tools[].name' ``` Then the three-step handshake with the key. `initialize` returns an `Mcp-Session-Id` header that the next calls must send: ```bash H=(-H "Authorization: Bearer $BEACON_MCP_API_KEY" -H "Content-Type: application/json" -H "Accept: application/json, text/event-stream") SESSION=$(curl -s -D - -o /dev/null "${H[@]}" $BEACON/beacon/mcp \ -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"1"}}}' \ | awk -F': ' 'tolower($1)=="mcp-session-id"{print $2}' | tr -d '\r') curl -s "${H[@]}" -H "Mcp-Session-Id: $SESSION" -H "MCP-Protocol-Version: 2025-06-18" $BEACON/beacon/mcp \ -d '{"jsonrpc":"2.0","method":"notifications/initialized"}' curl -s "${H[@]}" -H "Mcp-Session-Id: $SESSION" -H "MCP-Protocol-Version: 2025-06-18" $BEACON/beacon/mcp \ -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' ``` - [ ] `initialize` returns a non-empty `$SESSION`. - [ ] `tools/list` returns the tools above (plus any `q_…` saved-query tools your project exposes). ## If something goes wrong | Symptom | Cause and fix | |---|---| | `401` with a `WWW-Authenticate` header | No or invalid key. The variable is empty in the client's environment, the key expired or was revoked, or the header is not `Authorization: Bearer sk-sem_…`. | | `403` at the door, or "Permission denied: this credential does not have the Execute scope" | The key has only `Read`, or you no longer hold the Editor role. Create an `Execute` key as a user who does. | | "No projects are available to you." | None of the key's projects is one you may still use. Check your project membership. | | "Multiple projects available (IDs: …)" | The key reaches several projects. Pass `project_id` on every call, or create a key limited to one project. | | "Access denied: you may read project N but not run ad-hoc queries in it." | You are a Viewer there. `ask`, `query` and `dry_run` need Editor. | | 400 "A new session can only be created by an initialize request" | A curl call skipped `initialize`, or `$SESSION` is empty. Rerun the handshake from step 1. | | The client shows no tools but the key works in curl | The client did not start with the variable. Start it from the shell where `BEACON_MCP_API_KEY` is set. | ## Next - [Ask your data questions with an AI agent](https://moberghr.github.io/beacon/cookbooks/ask-your-data/): the tool workflow and prompts that get good answers. - [Give an AI agent access to one project only](https://moberghr.github.io/beacon/cookbooks/agent-one-project/): restricted projects, masking and the audit. - [MCP server](https://moberghr.github.io/beacon/mcp/server/): every tool, argument and guardrail. - [API keys](https://moberghr.github.io/beacon/security/api-keys/): scopes, expiry and revocation. --- # Add Beacon to claude.ai and Claude Desktop with Entra sign-in > Register Beacon's MCP endpoint as a custom connector in Claude, with people signing in through Microsoft Entra ID and Beacon mapping each person to projects, and check the sign-in path with beacon-doctor. Source: https://moberghr.github.io/beacon/cookbooks/claude-connector-entra/ | | | |---|---| | **You end up with** | A custom connector in claude.ai and Claude Desktop that signs people in with Entra and gives them Beacon's tools in the projects you map them to | | **Time** | about 45 minutes, plus tenant admin consent | | **You need** | A running Beacon reachable over HTTPS (`Beacon:PublicBaseUrl` set, see [MCP and AI clients](https://moberghr.github.io/beacon/mcp/connect/#public-url-and-discovery)), Entra rights to create app registrations and grant admin consent, a project with data sources, and permission in Claude to add custom connectors (an organization owner may have to add the connector first) | | **Applies to** | Beacon 5.0 · Full Beacon (a Beacon with data sources, mapping Entra callers to projects). For your own application's endpoints as tools, use the [MCP gateway](https://moberghr.github.io/beacon/mcp/gateway/) instead | :::tip[Let your AI do it] This recipe is half Entra portal and half Beacon configuration, so ask your agent to **guide you** through it and to make only the repository changes. Paste this into your coding agent. ```text wrap Read https://moberghr.github.io/beacon/cookbooks/claude-connector-entra.md and guide me through it step by step. You may edit this repository's Beacon configuration and code. For everything in the Entra portal or in Claude, tell me exactly what to click and what value to enter, then wait for me to confirm. Never change tenant settings yourself, never ask me to paste a client secret or token into the chat, and keep secrets in user secrets or my secret store. Ask me before every decision the recipe marks as one, and show me the result of each "Check it works" item. ``` ::: ## Before you start | Decision | Options | Default | |---|---|---| | Who may connect | Users holding an app role, members of groups, or every user of the tenant | An app role (`Beacon.Mcp.User`) | | Projects they reach | One project for everyone, or extra projects per group or role (`GroupProjects`) | One project | | Beacon user per person | Provisioned on first call (`AutoProvision`, role `DefaultRoleName`) or none | Provisioned, role `Viewer` | | What they can run | `Viewer` reads the catalog and runs saved-query tools only; `ask`, `query` and `dry_run` need Editor | Raise `DefaultRoleName` to `Editor` only if people should run SQL | Provisioned people get no project membership. For a restricted project, add each of them as a member, or they reach nothing in it. See [What a caller reaches](https://moberghr.github.io/beacon/mcp/entra-callers/#what-a-caller-reaches). ## 1. Create the API app registration This is the registration whose tokens Beacon validates. Use your Beacon's existing one if there is one. 1. **Manifest**: set `api.requestedAccessTokenVersion` to `2`. Do this first; Entra accepts an HTTPS Application ID URI only for a v2 app. 2. **Expose an API → Application ID URI**: keep `api://` and add the exact MCP URL, `https://beacon.example.com/beacon/mcp`, to `identifierUris`. No trailing slash. Claude sends that URL as the `resource` parameter, and a mismatch fails with `AADSTS9010010`. Entra may require the host name to be a verified domain of the tenant. 3. **Expose an API → Add a scope**: `mcp.access`, delegated. Clients request it as `api:///mcp.access`. 4. **App roles**: create `Beacon.Mcp.User` for users or groups, and assign it to the people who may connect (**Enterprise applications → Users and groups**). The tokens Claude receives are v2 access tokens: `iss` is `https://login.microsoftonline.com//v2.0`, `aud` is the API's client id GUID, `scp` is `mcp.access`. ## 2. Create a client registration for Claude Entra has no Dynamic Client Registration, so Claude cannot register itself. 1. **New registration**, then **Authentication → Add a platform → Web**, redirect URI `https://claude.ai/api/mcp/auth_callback`. 2. **Certificates & secrets**: create a client secret and note its expiry. The connector stops working when it expires. Keep the value for step 4; do not commit it. 3. **API permissions → Add a permission → APIs my organization uses →** your API registration → **Delegated** → `mcp.access`. 4. **Grant admin consent** for the tenant. ## 3. Configure Beacon Bearer validation must name issuers and audiences once any caller is configured; the host refuses to start otherwise. ```csharp builder.Services.AddBeaconJwtAuthentication(x => { x.EnableBearerAuthentication = true; x.Validation.JwksEndpoint = "https://login.microsoftonline.com//discovery/v2.0/keys"; x.Validation.ValidIssuers = ["https://login.microsoftonline.com//v2.0"]; x.Validation.ValidAudiences = ["", "api://"]; }); ``` ```json { "Beacon": { "PublicBaseUrl": "https://beacon.example.com", "Mcp": { "ProtectedResource": { "AuthorizationServers": [ "https://login.microsoftonline.com//v2.0" ], "ScopesSupported": [ "api:///mcp.access", "offline_access" ] }, "Callers": { "AllowedTenants": [ "" ], "Users": { "Enabled": true, "RequiredRoles": [ "Beacon.Mcp.User" ], "ProjectIds": [ 4 ], "Scope": "Execute", "AutoProvision": true, "DefaultRoleName": "Viewer" } } } } } ``` | Key | Why it matters here | |---|---| | `Beacon:PublicBaseUrl` | The `resource` Claude sends is built from it: `https://beacon.example.com/beacon/mcp`. Without it, Beacon uses the request's `Host` header. | | `Mcp:ProtectedResource:AuthorizationServers` | Published in `/.well-known/oauth-protected-resource`. Use the **v2** issuer. Instead of listing it you can set `DeriveAuthorizationServerFromJwtIssuer` to `true`. | | `Mcp:ProtectedResource:ScopesSupported` | Without it Beacon publishes `Execute`, which Entra does not know. `offline_access` gives Claude a refresh token; without it people sign in again every hour, and Beacon logs a startup warning. | | `Mcp:Callers:AllowedTenants` | Required and non-empty once `Users:Enabled` is on. | | `Mcp:Callers:Users:RequiredRoles` / `RequiredGroups` | With neither set, startup fails unless `AllowAnyTenantUser` is `true`. Prefer app roles: Entra leaves the `groups` claim out for users in more than about 200 groups. | | `Mcp:Callers:Users:DefaultRoleName` | Must not be Admin or a functional role; startup fails otherwise. | Keep the app's own project list small: every accepted user gets `ProjectIds`. Add `GroupProjects` for more. All keys are described in [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/#configuration). Deploy, then check the two discovery documents from any machine: ```bash curl -s https://beacon.example.com/.well-known/oauth-protected-resource | jq ``` - [ ] `resource` is `https://beacon.example.com/beacon/mcp`. - [ ] `authorization_servers[0]` is the v2 issuer and `scopes_supported` lists your scope and `offline_access`. ## 4. Add the connector in Claude Claude's menus change; if a label below differs, follow Anthropic's current custom-connector documentation. Beacon needs only the URL and the client registration's id and secret. 1. In claude.ai, open **Settings → Connectors** and **Add custom connector**. An organization owner may have to add it for the organization first. 2. Enter the server URL `https://beacon.example.com/beacon/mcp`. 3. In **Advanced settings**, enter the **OAuth Client ID** and **OAuth Client Secret** of the registration from step 2. 4. Connect. Claude opens the Entra sign-in; the person consents, or admin consent already covers it. Claude Desktop uses the connectors of the claude.ai account it is signed in to. ## Check the sign-in path with beacon-doctor `beacon-doctor` checks the endpoint the way a connector sees it: the 401 challenge, the protected-resource metadata, the authorization server's metadata and signing keys, PKCE and `offline_access`. ```bash dotnet tool install -g Moberg.Beacon.Doctor beacon-doctor mcp https://beacon.example.com/beacon/mcp ``` Each line starts with `PASS`, `WARN` or `FAIL`; the exit code is `1` when any check fails and `0` otherwise. Add `--json` for machine-readable output. With an Entra access token for your API (`--token `, or the `BEACON_MCP_TOKEN` environment variable) it also tries `initialize` and `tools/list`, which proves the token maps to a caller. The token is sent to that endpoint only. Without one, the `handshake` line warns that it was not tried. ## Check it works - [ ] `beacon-doctor` reports no `FAIL`. A `WARN` for `offline_access` means sign-in will repeat hourly: add it to `ScopesSupported`. - [ ] In Claude, the connector shows as connected after the Entra sign-in, and lists Beacon's tools. - [ ] Ask in a new chat: *"Use Beacon's get_context to describe my project."* The answer names your project. - [ ] In Beacon, a new user exists for the person (when `AutoProvision` is on), and their calls appear in the MCP audit (see [MCP observability and audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/#reading-the-audit)). ## If something goes wrong | Symptom | Cause and fix | |---|---| | `AADSTS9010010` at sign-in | The `resource` Claude sent is not an Application ID URI of the API registration, or a requested scope belongs to another registration. Compare `resource` in `/.well-known/oauth-protected-resource` with `identifierUris`, character for character. | | `401` with `error="invalid_token"` | Beacon rejected the token. Compare `iss` and `aud` with `ValidIssuers` and `ValidAudiences`; a v2 token's `aud` is the client id GUID. | | `403` after sign-in | The token is valid but the person is not an accepted caller: check `Users:Enabled`, `RequiredRoles` or `RequiredGroups` (is the role assigned to them?) and `AllowedTenants`. | | "Access denied: you may read project N but not run ad-hoc queries in it." | The provisioned user is a `Viewer`. Raise `DefaultRoleName` or the user's role to Editor, and for a restricted project add them as an Editor member. | | "No projects are available to you." | `ProjectIds` is empty for this person, or the project is restricted and they are not a member. | | `503` with `Retry-After` | Beacon could not fetch the Entra signing keys and has none cached. Check outbound access to `login.microsoftonline.com`. | | Sign-in repeats every hour | `offline_access` is missing from `ScopesSupported`. | | Connector stopped working months later | The client secret in step 2 expired. Create a new one and update the connector. | | `beacon-doctor` warns about PKCE | Entra's metadata does not advertise PKCE although it supports it. Whether Claude enforces that is for Anthropic's documentation to say; the warning is not a Beacon fault. | ## Next - [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/): system callers, token classification, replacing the mapper. - [Ask your data questions with an AI agent](https://moberghr.github.io/beacon/cookbooks/ask-your-data/): what to say once the connector works. - [MCP gateway](https://moberghr.github.io/beacon/mcp/gateway/): the same connector for your own application's endpoints, with no data sources. - [Give an AI agent access to one project only](https://moberghr.github.io/beacon/cookbooks/agent-one-project/): limiting what a caller reaches. --- # Ask your data questions with an AI agent > Use Beacon's MCP tools in the order that gives correct, checked answers: orient, find, ground, validate, run, explain and rate, with prompts you can paste and instructions to give your agent. Source: https://moberghr.github.io/beacon/cookbooks/ask-your-data/ | | | |---|---| | **You end up with** | An AI client that answers questions about your data by following Beacon's tool workflow, validates its SQL before running it, and records whether an answer was right | | **Time** | about 15 minutes | | **You need** | An AI client already connected to Beacon ([coding agent](https://moberghr.github.io/beacon/cookbooks/connect-coding-agent/) or [Claude connector](https://moberghr.github.io/beacon/cookbooks/claude-connector-entra/)) with Execute scope, and the Editor role in the project (a Viewer reads the catalog but cannot run SQL) | | **Applies to** | Beacon 5.0 · Full Beacon | :::tip[Let your AI do it] This recipe is for an AI client that is already connected to Beacon (Claude, Claude Desktop, Cursor), not for a coding agent working in a repository. Paste this into that client's chat. ```text wrap You are connected to Beacon's MCP tools. Follow https://moberghr.github.io/beacon/cookbooks/ask-your-data.md : start with get_context, find tables with search, ground yourself with get_query_context, validate every SQL statement with dry_run before query, and only use ask when I want a one-shot answer. Tell me which tool you are calling and why. Never invent table or column names. ``` ::: ## Before you start | Decision | Options | Default | |---|---|---| | Who writes the SQL | Your agent (`get_query_context` → `dry_run` → `query`), or Beacon (`ask`) | The agent, when you need to see and control the SQL; `ask` for quick business questions | | Which project | One, or several | If the credential reaches several projects, the agent passes `project_id` on every call | ## How the tools fit together Beacon sends this workflow to clients at `initialize` as server instructions, so most clients already know it. The tools, in order: | Step | Tool | Use it to | Key arguments | |---|---|---|---| | 1 | `get_context` | See the project's data sources, tables, quality and documentation status | `project_id`, `format` (`overview` or `agents_md`) | | 2 | `search` | Find tables, columns and documents by keyword | `query`, `max_results` (default 20, max 50), `offset` | | 2 | `get_documentation` | Read the documentation of a data source or one table | `datasource_name`, `table_name`, `schema_name`, `response_format` (`concise` or `detailed`), `document` | | 3 | `get_query_context` | Get schemas with sample values, join paths, verified examples and glossary terms ranked for one question | `question`, `datasource_name` or `datasource_id`, `max_chars` | | 4 | `dry_run` | Check SQL through every gate without running it | `sql`, `datasource_name` or `datasource_id` | | 5 | `query` | Run the validated SELECT | `sql`, `datasource_name` or `datasource_id`, `max_rows` (default 100) | | alt | `ask` | Let Beacon route, write and run the SQL | `question`, `execute` (default `true`) | | after | `feedback` | Record that an `ask` answer was `correct` or `incorrect` | `signal_id`, `verdict`, `corrected_sql`, `note` | Every tool also takes an optional `project_id`. Full argument tables: [MCP server](https://moberghr.github.io/beacon/mcp/server/#tools). If a reviewed saved query already answers the question, an approved `q_` tool is better than any hand-written SQL; see [Publish an approved saved query as an MCP tool](https://moberghr.github.io/beacon/cookbooks/saved-query-tool/). ## 1. Orient in a project ```text wrap title="Say to Claude" Use Beacon's get_context to describe my project: which data sources it has, the main tables, and how well they are documented. Do not run any queries yet. ``` `get_context` is audit-only and reads no rows. Look at the documentation status it reports: a table without documentation gives the agent less to work with. ## 2. Find where data lives ```text wrap title="Say to Claude" Where does Beacon keep information about refunds? Use search for "refund" and "chargeback", then read the documentation of the two most likely tables with get_documentation. Tell me which columns hold the amount and the date. ``` `search` matches keywords, and meaning too when a local embedder is configured. `get_documentation` with `table_name` returns columns, types, relationships and quality rules. ## 3. Write and validate SQL before running it ```text wrap title="Say to Claude" I want monthly refund totals for the last six months. Call get_query_context with that question, write the SQL yourself from what it returns, run it through dry_run, and show me the SQL and the dry_run verdict. Do not call query until I say go. ``` `get_query_context` returns real join paths and verified examples. Sections marked **(authoritative)** are human-verified. `dry_run` runs the read-only gate, a schema column check and the database's own validation, and reports every issue at once. It never executes the statement. When the verdict is valid it returns the exact SQL that would run, with the row limit applied. Then: ```text wrap title="Say to Claude" Go. Run that SQL with query, and tell me how many rows came back and whether the result was truncated. ``` If the result says more rows exist, raise `max_rows` or narrow the question: results are capped at the project's maximum row limit. ## 4. Ask a one-shot business question ```text wrap title="Say to Claude" Use Beacon's ask tool: "How many orders were placed last week, by country?" Show me the SQL it generated. ``` `ask` picks the data source, writes SQL grounded in your schema, runs it and ends its answer with a `_signal_id: N_` marker. Pass `execute: false` to get the SQL without running it. A question that spans data sources is joined in memory. Use `ask` for speed and the step-by-step flow when the SQL itself matters. ## 5. Explain a number ```text wrap title="Say to Claude" Explain where the "active customers" figure of 12,430 came from. Show the SQL behind it, the tables it reads, and any filter that decides who counts as active. If you cannot find it in Beacon, say so rather than guessing. ``` The agent can answer from the generated SQL of an `ask` it made, or by searching the documentation and glossary. If Beacon has no definition, a good answer says that. ## 6. Rate the answer ```text wrap title="Say to Claude" That answer is correct. Record it with Beacon's feedback tool using the signal_id from the ask response. ``` or, when you fixed the SQL: ```text wrap title="Say to Claude" That answer is wrong: it counted cancelled orders. Record it as incorrect with feedback, using this corrected SQL: . ``` An `incorrect` verdict keeps the answer out of the patterns Beacon learns. A `correct` one puts it on a list where an Admin may promote it to a verified example; feedback never promotes it by itself. Corrected SQL is stored only when it passes Beacon's checks and the project allows the feedback tool to store text. Only add a `note` when you ask for one. Details: [`feedback`](https://moberghr.github.io/beacon/mcp/server/#feedback). ## Write it into your agent's instructions Put this in `CLAUDE.md`, `AGENTS.md` or your client's custom instructions so every session starts with the workflow. `get_context` with `format: "agents_md"` generates a project brief you can add below it. ```md ## Beacon data tools - Start with `get_context`. Use `search` and `get_documentation` to find tables before writing SQL. - Write your own SQL like this: `get_query_context` for the question, then `dry_run`, then `query`. Never call `query` with SQL that `dry_run` has not accepted. Do not invent table or column names. - If an approved `q_` saved-query tool answers the question, call it instead of writing SQL. - Use `ask` only for quick one-shot questions; show the generated SQL with the answer. - Pass `project_id` on every call when the credential reaches more than one project. - Beacon is read-only. Do not try to write, alter or drop anything, and do not try to read other projects. - After a user confirms an `ask` answer is right or wrong, record it with `feedback` and the `signal_id`. Only include a `note` when the user asks for one. - Quote the SQL and row counts you used. If a result is truncated or masked, say so. ``` ## What the agent cannot do These are enforced by Beacon, not by the prompt. See [Safety and guardrails](https://moberghr.github.io/beacon/mcp/server/#safety--guardrails). | It cannot | Why | |---|---| | Write, alter or drop data | Only a single read-only statement passes the gate; `dry_run` shows which layer refused it. | | Reach another project | A project outside the credential's list answers "Access denied: you do not have access to project N." | | See unmasked PII | Columns detected as PII are masked in results; when Beacon cannot prove where a PII column went, it masks every column. | | Read the engine's catalog or another database | SQL naming another database, or reading `information_schema`, `pg_catalog` or similar, is refused (`cross_database`, `system_catalog`). | | Run SQL as a Viewer | `ask`, `query`, `dry_run` and `feedback` need Editor in the project. | | Return unlimited rows or run forever | Rows, result size, statement time and concurrent queries are capped per project. | | Teach Beacon on its own authority | `feedback` records a verdict; only an Admin promotes a verified example. | ## Check it works - [ ] `get_context` describes your project's data sources. - [ ] `dry_run` on a valid SELECT reports a valid verdict and the SQL with a row limit; on `DELETE FROM x` it reports an issue and nothing runs. - [ ] `query` returns rows, with PII columns masked when the project has detection on. - [ ] `ask` ends with `_signal_id: N_`, and `feedback` with that number answers that the verdict was recorded. - [ ] Each call appears in the MCP audit (see [Give an AI agent access to one project only](https://moberghr.github.io/beacon/cookbooks/agent-one-project/#see-what-the-agent-did)). ## If something goes wrong | Symptom | Cause and fix | |---|---| | "Access denied: you may read project N but not run ad-hoc queries in it." | The caller is a Viewer. Give the owner the Editor role (and Editor membership in a restricted project). | | "Multiple projects available (IDs: …)" | Pass `project_id` on every call, or use a key limited to one project. | | `dry_run` says the schema gate could not run | The data source has no extracted metadata yet, so the column check could not run and the verdict is invalid. Have the data source's metadata extracted, then try again. | | `dry_run` reports `cross_database` or `system_catalog` | The SQL names another database or reads the engine catalog. Rewrite it against the data source's own tables. | | `ask` answers "AI features not configured" | No LLM provider is configured on this Beacon. Use `get_query_context`, `dry_run` and `query` instead. | | Answers miss joins or misread a term | Beacon only knows what it was given. Register the join, add the glossary term, or promote a correct answer; see [Knowledge base](https://moberghr.github.io/beacon/mcp/knowledge-base/). | | Result is cut off | The result says it is truncated. Raise `max_rows` (up to the project's maximum) or narrow the question. | ## Next - [Prompt library](https://moberghr.github.io/beacon/cookbooks/prompts/): more prompts for specific jobs. - [Publish an approved saved query as an MCP tool](https://moberghr.github.io/beacon/cookbooks/saved-query-tool/): fixed, reviewed SQL for recurring questions. - [Ground Beacon with your docs](https://moberghr.github.io/beacon/cookbooks/ground-with-your-docs/): better answers from better documentation. - [MCP server](https://moberghr.github.io/beacon/mcp/server/): every tool and setting. --- # Publish an approved saved query as an MCP tool > Turn a reviewed, parameterized Beacon query into a fixed MCP tool such as q_loan_book_by_month, so agents call approved SQL instead of writing their own. Source: https://moberghr.github.io/beacon/cookbooks/saved-query-tool/ | | | |---|---| | **You end up with** | A saved query that agents call as the tool `q_`, always running the approved SQL with their arguments bound as database parameters | | **Time** | about 20 minutes, plus the reviewer's time | | **You need** | A query with at least one step you can edit, a second person with the **Approver** permission, a **SecurityAdmin** or an Admin who can read the query, and the [approval workflow](https://moberghr.github.io/beacon/security/approvals/) turned on by the host | | **Applies to** | Beacon 5.0 · Full Beacon | :::tip[Let your AI do it] Most of this is done in Beacon's UI by different people (author, approver, SecurityAdmin), so ask your agent to **guide you** through it and check each result. Paste this into your agent. ```text wrap Read https://moberghr.github.io/beacon/cookbooks/saved-query-tool.md and guide me through it step by step. Tell me which Beacon screen to open and what to enter, then wait for me to confirm. Ask me before every decision the recipe marks as one. Never approve a change for me, and never enable the tool yourself with an account that wrote or approved the query. Show me the result of each "Check it works" item. ``` ::: ## Before you start | Decision | Options | Default | |---|---|---| | Tool name | 3 to 41 characters: `a-z`, `0-9`, `_`, starting with a letter. Agents see `q_` | A short noun phrase such as `loan_book_by_month` | | Parameters | `Number`, `DateTime` or `String`, each with a description | Describe each one for the agent: the description becomes the argument's description | | Who enables it | A SecurityAdmin, or an Admin, who can read the query | With the approval workflow on, not the person who requested or approved the version | | Strict mode | `ApprovalWorkflow.RequireApprovedVersionToRun` on or off | On for regulated deployments: the approval must name the exact content hash | Every argument is required; the query model has no defaults. Only database data sources run as tools, not API sources. ## 1. Write the query with parameters Create or open the query and write the step SQL with `{name}` placeholders: ```sql SELECT date_trunc('month', booked_at) AS month, sum(principal) AS principal FROM loans WHERE booked_at >= {from} AND booked_at < {to} GROUP BY 1 ORDER BY 1 ``` Declare each parameter on the step with a type and a description (`from`: DateTime, "First day to include, ISO 8601"). The reserved name `project_id` cannot be used, a parameter must have a placeholder, and the same name must have the same type in every step. Run it with the editor's preview to check the SQL, then save. ## 2. Get a version approved With the approval workflow on, saving SQL files a version for approval and does not change the live query. 1. The author saves. The query shows that it awaits approval. 2. A different person with the **Approver** permission opens **Approvals** (`/beacon/approvals`), reads the step diff, optionally leaves a **Reviewer comment**, and chooses **Approve**. 3. The requester cannot approve their own change, and neither can anyone else who shaped it. Admins are not exempt. Only the query's **active** version, with an approval for exactly that version, can run as a tool. A later edit is pending again; the tool keeps running the approved version until the edit is approved. See [Which version runs](https://moberghr.github.io/beacon/mcp/saved-query-tools/#which-version-runs). ## 3. Expose it Do this as a SecurityAdmin (or an Admin) who is not the requester or approver of the version. An Admin may publish any query; anyone else only a query they can read (`404` otherwise). **In the UI.** Open the query's detail page and find the **MCP tool** card. It shows the approved version that calls will run (`v4`, for example) or why none can. Tick **Expose as MCP tool**, enter the **Tool name** (the card shows `q_`), optionally a **Description** (what the tool answers, as the agent reads it; up to 1000 characters), and **Save**. The status pill changes to **Exposed**. **Through the API**, as the same signed-in user: ```http PUT /beacon/api/queries/42/mcp-tool Content-Type: application/json { "enabled": true, "name": "loan_book_by_month", "description": "Outstanding loan book per month for a date range." } ``` The name is unique across all live queries. Enabling is refused with `400` when the name is taken or invalid, or when no approved version can run, and with `403` when you requested or approved the version while the workflow is on (unless the host allows self-approval). The query detail (`GET /beacon/api/queries/{id}`) carries `mcpToolEnabled`, `mcpToolName`, `mcpToolRunnableVersionNumber` and `mcpToolIssue`. The issue explains why a tool cannot be called: no approved version, a parameter shape that cannot be a tool, or no project that holds every data source the query reads. Disable it the same way with `"enabled": false`. Anyone who may publish can disable. ## 4. What the agent sees The tool appears in `tools/list` for callers whose project contains **every** data source the query reads and whose project role may run saved SQL (Viewers may). - The name `q_loan_book_by_month`, the description, and the approved version number. - An input schema built from the parameters: `number` for Number, an ISO 8601 `string` for DateTime, a `string` (up to 4000 characters) for String. Every parameter is required, and extra arguments are refused. - Annotations `readOnlyHint: true`, `destructiveHint: false`. A call returns a markdown table plus structured content: `columns`, `rows`, `rowCount`, `truncated` and the `versionNumber` that ran. **Many tools.** Up to 25 tools visible to one caller are listed by name (`SavedQueryTools.NamedToolLimit`, set in `AddBeaconServices(..., options => options.SavedQueryTools.NamedToolLimit = 40)`). Above that, the caller gets two tools instead: | Tool | Arguments | |---|---| | `search_saved_queries` | `query` (keywords; empty lists all), `limit` (default 10, max 50) | | `run_saved_query` | `name` (required), `arguments` (matching the input schema, with `project_id` when the schema lists one) | When a credential reaches the same tool through several projects, the schema adds `project_id` and calls must pass it. ## Check it works - [ ] The **MCP tool** card shows **Exposed** and the version number you approved. - [ ] From a connected client (see [Connect Claude Code, Cursor or VS Code](https://moberghr.github.io/beacon/cookbooks/connect-coding-agent/)), `tools/list` includes `q_loan_book_by_month` (or `search_saved_queries` above the limit). - [ ] Ask the agent: *"Call q_loan_book_by_month with from 2026-01-01 and to 2026-07-01."* It returns rows and the `versionNumber` you approved. - [ ] Edit the query without approving: the tool still returns the old version's numbers. - [ ] The call is in the MCP audit as `q_loan_book_by_month`. Saved-query calls are audit-only and record no learning signals. ## If something goes wrong | Symptom | Cause and fix | |---|---| | **Expose as MCP tool** is disabled, or `400` "cannot be exposed" | No approved active version, or the parameters cannot form a tool. The card names the reason. | | `403` when enabling | You requested or approved the version the tool would run. Ask a different SecurityAdmin or Admin. | | `403` for a non-SecurityAdmin | Publishing needs the SecurityAdmin permission or Admin. | | `404` when enabling | You cannot read the query, or it does not exist. | | Tool enabled but not listed | The approved version is not the active one, or no project in the caller's reach holds every data source the query reads. The card's `mcpToolIssue` says which. | | "The tool name '…' is already used" | Names are unique across live queries. Pick another. | | Call fails with a missing or unknown argument | Every parameter is required and nothing else is accepted; dates must be ISO 8601. | | Strict mode will not run an old tool | The approval predates content hashes. Restore the version and approve it again. | | `truncated: true` | More rows than the project's MCP row limit. Narrow the arguments or raise the limit in **MCP settings**. | ## Next - [Saved query tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/): the security model, limits and a routine example. - [Approvals](https://moberghr.github.io/beacon/security/approvals/): who approves, strict mode and the API. - [Ask your data questions with an AI agent](https://moberghr.github.io/beacon/cookbooks/ask-your-data/): when to prefer a saved-query tool over generated SQL. --- # Give an AI agent access to one project only > Confine an agent to a single restricted project with a project-limited Execute key, set that project's masking and row limits, verify the agent cannot reach anything else, and read what it did in the MCP audit. Source: https://moberghr.github.io/beacon/cookbooks/agent-one-project/ | | | |---|---| | **You end up with** | An API key that reaches one project and nothing else, masked and row-capped results in that project, and an audit trail of every call the agent made | | **Time** | about 20 minutes | | **You need** | An Admin or UserAdmin (to restrict the project and add members), a project with its data sources linked, a Beacon user for the agent's key to act as, and an Auditor or Admin to read the audit | | **Applies to** | Beacon 5.0 · Full Beacon | :::tip[Let your AI do it] Most of this is done in Beacon's UI under different roles, so ask your agent to **guide you** and to check each result. Paste this into your agent. ```text wrap Read https://moberghr.github.io/beacon/cookbooks/agent-one-project.md and guide me through it step by step. Tell me which Beacon screen to open and what to enter, then wait for me to confirm. Ask me before every decision the recipe marks as one. Never ask me to paste the API key into the chat. Show me the result of each "Check it works" item. ``` ::: ## Before you start | Decision | Options | Default | |---|---|---| | Whose identity the key uses | A dedicated Beacon user for the agent, or your own | A dedicated user: the key can never reach more than its owner does, so a small owner means a small key | | The owner's role | Editor (the agent may run SQL) or Viewer (catalog and saved-query tools only) | Editor in this project, Viewer elsewhere | | Masking | PII detection on, plus custom patterns for your own identifiers | On; it is pinned on by `Beacon:Mcp:ForcePiiDetection` unless your host sets it to `false` | | Row cap | A project maximum below the global 1000 | The lowest value the agent needs | ## 1. Use a restricted project A restricted project exists only for its members and Admins; to anyone else it answers `404`. New projects are created restricted, with the creator as project admin. 1. Create the project (**Projects → New project**), or open an existing one. 2. On the project page, the **Access** card (UserAdmin or Admin) shows **Open** or **Restricted**. Choose **Restricted** and confirm. Add members before restricting an existing project, or its users lose access. 3. **Link data sources** with only the sources the agent needs. A resource reading several data sources is reachable only through a project that holds all of them. 4. On the **Members** tab, add the agent's Beacon user with the **Editor** role (or **Viewer** to limit it to the catalog and saved-query tools). :::caution Project access decides who may run SQL through Beacon, not what the database login can read. Give each data source a login that sees only what this project should see. Beacon also treats a data source that another open project links as reachable through that project; check the **Access** card's list of projects that share a source. ::: ## 2. Create a key limited to that project Sign in as the agent's user (keys are created in a signed-in browser session, never with another key), open **API Keys** (`/beacon/api-keys`) and choose **Generate key**: - **Key name**: for example `Reporting agent`. - **Scopes**: **Execute**. The MCP endpoint requires it; it needs the Editor role or above on the owner. - **Projects**: only this project. The picker lists projects the owner can read. - **Expiration date**: a short one. The default is 90 days. Copy the key (`sk-sem_…`) once, and store it in your secret store or an environment variable; see [Connect Claude Code, Cursor or VS Code](https://moberghr.github.io/beacon/cookbooks/connect-coding-agent/) for wiring it into a client. What the key can reach is the **intersection** of its project list and what its owner may still use, at no more than the owner's role there, and never the Admin bypass. Remove the owner's membership, and the key reaches nothing. See [Project restrictions](https://moberghr.github.io/beacon/security/api-keys/#project-restrictions). ## 3. Set the project's masking and limits Open **MCP settings** (`/beacon/mcp-settings`), switch the scope selector at the top from the global defaults to your project, and tick **Override** on each field you want to set for this project only: | Field (tab) | Set it to | Effect | |---|---|---| | **Max row limit** (Guardrails) | e.g. 200 | Ceiling on rows `query`, `ask` and saved-query tools return in this project. Must be at least 1. | | **Enable PII detection** and **Custom PII patterns** (Guardrails) | On, plus one regex per line | Masks detected PII columns in results. Add patterns for identifiers Beacon cannot detect. | | **Enforce read-only queries** (Guardrails) | On | Keep on; with `ForceReadOnly` (default) it cannot be turned off. | | **Statement timeout (s)**, **Max result size (bytes)** (Execution) | e.g. 15 and 65536 | Bounds each statement and its payload. | | **Max concurrent queries per API key** (Execution) | e.g. 2 | Limits parallel calls from this key. | | **Retain question and SQL text in signals and audit** (Execution) | Your policy | Whether the audit keeps what the agent asked and ran. Off keeps the audit structural only. | Editing needs the SecurityAdmin permission (an Auditor can read it). Deployment locks and ceilings in `Beacon:Mcp` can pin or cap these values from configuration; a locked field is hidden or disabled and says so. When a data source is shared with other projects, the strictest settings among them apply. See [Safety and guardrails](https://moberghr.github.io/beacon/mcp/server/#safety--guardrails). ## 4. See what the agent can and cannot reach Connect a client with the key and try, in this order: | Ask the agent | What happens | |---|---| | "Run `get_context`." | Describes this project. With a single-project key no `project_id` is needed. | | "Run `get_context` with `project_id` 99" (another project) | "Access denied: you do not have access to project 99." The call is audited. | | "Query the customers table." | Rows return with PII columns masked, capped at the row limit; the result says when it was truncated. | | "Run `DELETE FROM customers`" | Refused by the read-only gate. `dry_run` shows which layer refused it. | | "Query `information_schema.tables`" or another database | Refused (`system_catalog`, `cross_database`). | The key cannot change settings, manage keys, members or data sources: API keys never reach administrative routes, and they hold no functional permission. ## See what the agent did Every tool call, including refusals and errors, is written to the MCP audit with the key (`ApiKeyId`), its owner, the tool, the project, timing, row counts and the statements it ran (`sql` only when the project retains content). An Auditor or an Admin reads it: ```http GET /beacon/api/mcp/audit?from=2026-10-01T00:00:00Z&to=2026-10-08T00:00:00Z&projectId=4&tool=query&page=1&pageSize=100 ``` `from` and `to` are required (UTC, at most 93 days apart); `projectId`, `tool`, `callerHash` and `userId` filter. The rows are not filtered by project membership, so give the role only to people who should see every project's calls. An Auditor or Admin can also read it on the **MCP Audit** page (`/beacon/mcp-audit`), which calls this endpoint. Reading the audit is itself recorded in the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/), where creating and revoking keys is also logged. Details: [Reading the audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/#reading-the-audit). To stop the agent at once, **Revoke** the key on **API Keys**. It takes effect on the next request. Disabling the owner revokes all of their keys. ## Check it works - [ ] The project page shows **Restricted**, and a user who is not a member gets "not found" for it. - [ ] The key lists exactly one project on **API Keys**. - [ ] A call naming another project is refused with "Access denied: you do not have access to project N." - [ ] A result from `query` has PII columns masked and no more rows than **Max row limit**. - [ ] A write statement is refused. - [ ] The audit export shows the calls above, including the refused ones, for this key. ## If something goes wrong | Symptom | Cause and fix | |---|---| | "No projects are available to you." | The key's project is not one its owner may still use: the owner is not a member of the restricted project, or was disabled. Add the membership. | | "Access denied: you may read project N but not run ad-hoc queries in it." | The owner is a Viewer there. Make them an Editor member for `ask`, `query` and `dry_run`. | | "Permission denied: this credential does not have the Execute scope" | The key is `Read`, or its owner lost the Editor role. | | Cannot tick **Execute** when creating the key | The signed-in user is a Viewer. Execute keys need the Editor role or above. | | The **Access** card is missing | It is shown to UserAdmins and Admins only, and is hidden when the host manages membership. | | Restricting answers `409` | Project membership is unavailable in this deployment (the host manages it, or user management is off). | | A settings field cannot be changed | A deployment lock pins it, or a ceiling caps it; only the host's configuration lifts that. | | The key stopped working with a message about owner inactivity | The host sets `Beacon:ApiKeys:MaxOwnerInactivityDays` and the owner has not signed in for that long. A service account must sign in now and then. | | Audit export answers `403` | The caller is neither an Auditor nor an Admin. | ## Next - [Authorization](https://moberghr.github.io/beacon/security/authorization/): project roles, 404 versus 403, shared data sources. - [API keys](https://moberghr.github.io/beacon/security/api-keys/): scopes, expiry, owner changes. - [MCP observability and audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/): audit fields, retention, tracing. - [Audit to your SIEM](https://moberghr.github.io/beacon/cookbooks/audit-to-siem/): ship these events elsewhere. --- # Expose an endpoint as an MCP tool, safely, and test it > Turn one self-service endpoint of your ASP.NET Core application into a read tool and one into a write tool, bind the person's id from the caller, and pin the behaviour with an end-to-end test. Source: https://moberghr.github.io/beacon/cookbooks/endpoint-as-tool/ | | | |---|---| | **You end up with** | A read tool (`api_my_orders`) and a write tool (`api_request_leave`) on your MCP server, the person's id filled from the caller, refusals that reach the agent, and a test that lists the tools, calls one and checks another person's data is refused | | **Time** | about 30 minutes | | **You need** | An ASP.NET Core application with an MCP gateway ([MCP gateway](https://moberghr.github.io/beacon/mcp/gateway/)) or full Beacon with `AddHostEndpointTools` ([Host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/)); a PostgreSQL or SQL Server database with Beacon's schema migrated for the test | | **Applies to** | Beacon 5.0 · Both (gateway: `x.Tools(...)`; full Beacon: `AddBeaconMcp().AddHostEndpointTools(...)`) | :::tip[Let your AI do it] Paste this into Claude Code (or another coding agent) in your application's repository. ```text wrap Read https://moberghr.github.io/beacon/cookbooks/endpoint-as-tool.md and follow it step by step in this repository. Ask me which endpoint to expose and whether it is self-service before marking anything. Ask me before every decision the recipe marks as one. Never mark an admin endpoint, never commit secrets, and show me the result of each "Check it works" item. ``` ::: ## Before you start | Decision | Options | Default | |---|---|---| | Which endpoint | One that reads or changes the signed-in person's own records | A `/api/my/...` read | | Read or write | `ReadOnly = true`, or `ReadOnly = false` (needs `AllowWriteTools`) | Read | | How the person's id arrives | Bound from a claim of the call's principal (`BindFromCaller`) | Bound, never asked of the agent | | How the agent is held to self-service | Only mark self-service endpoints, and check `IsBeaconMcpCall()` where one endpoint also serves the browser | Mark only self-service endpoints | ## 1. Pick the endpoint and check it against the review list Mark an endpoint only when every line holds. If one fails, add a `/api/my/...` sibling that filters on the caller and takes no subject, and mark that. - [ ] **Self-service.** It reads or changes the person's own records. It is not an admin endpoint and is not gated by a permission a person may hold over others. - [ ] **Data minimisation.** It never returns other people's data to some callers (a manager's view, a list by sequential id). An agent reads free text and sends what it reads to a model. - [ ] **Authorization metadata.** It has `[Authorize]` or a policy. Startup refuses an endpoint with none, unless the application has a fallback policy. Never `[AllowAnonymous]`. - [ ] **Bound id.** The person's id is not an argument the agent supplies. - [ ] **Description.** Written for an agent (step 3). - [ ] **`ReadOnly` declared.** Always, on every tool. Beacon never infers it from the HTTP verb. ## 2. Bind the person's id from the caller An agent does not know the person's id in your application. Bind the argument to a claim of the principal the call runs as, so it leaves the tool's schema and carries the caller's own value. In the gateway, the claim is one your `IMcpHostUserResolver` returns in `McpHostUser.Claims`: ```csharp x.Tools(y => { y.AllowWriteTools = true; // only because step 4 marks a write tool y.BindFromCaller("userId", "app_user_id"); }); ``` In full Beacon, the same call is on the options of `AddHostEndpointTools`, and the claim comes from the `IMcpHostPrincipalFactory` you register (see [Who the call runs as](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/#who-the-call-runs-as)): ```csharp builder.Services.AddBeaconMcp() .AddHostEndpointTools(y => { y.ProjectName = "app"; y.AllowWriteTools = true; y.BindFromCaller("userId", "app_user_id"); }); ``` On every tool, a route, query, header or form value named `userId` (ignoring case), or a top-level property of a JSON body with that name, is removed from the schema. An agent that sends it anyway is refused with `Invalid arguments: userId is filled from the caller and cannot be sent.` A principal with no value for the claim refuses the call. A bound name that no tool takes is warned at startup, which usually means a typo. ## 3. Mark a read tool ```csharp using Beacon.MCP.HostEndpoints; app.MapGet("/api/my/orders", (int userId, OrderService orders) => orders.Of(userId)) .RequireAuthorization() .WithBeaconTool( "my_orders", "The signed-in user's own orders, newest first. Returns id, placedAt (UTC, ISO 8601), status " + "(pending, shipped, cancelled) and total (EUR). Takes no arguments. Returns an empty list when the user has no orders.", readOnly: true); ``` The name must match `^[a-z][a-z0-9_]{2,47}$` and be unique. MCP exposes it as `api_my_orders`. Write the description for the agent, not for a developer: - What comes back, field by field, with units and time zones. - What each argument means, and what leaving it out does. - Every refusal the endpoint can produce. Beacon appends the method and route to every description. Your endpoint still checks ownership as it does for the browser: the bound id is the caller's, and the endpoint decides what that person may see. ## 4. Mark a write tool A write tool is off until `AllowWriteTools` is on (step 2). Declare `ReadOnly = false` and state `Destructive` and `Idempotent`: ```csharp using Beacon.Core.HostEndpoints; [Authorize] [BeaconTool("request_leave", Description = "Request leave for the signed-in user. Arguments: from and to (dates, yyyy-MM-dd, inclusive) and " + "reason (optional). Returns the new request's id and status. Refused (400) when the dates overlap an existing " + "request. Call it only when the person asked for this change.", ReadOnly = false, Destructive = false, Idempotent = false)] [HttpPost("/api/time-off/requests")] public IActionResult RequestLeave([FromBody] RequestLeaveBody body) { /* validate, check balance, save */ } ``` `[BeaconTool]` goes on a controller action, or on a handler class whose framework copies class attributes into its endpoint (a Moberg Warp handler). On an MVC controller class it is refused at startup. A minimal API uses `.WithBeaconTool(name, description, readOnly: false, destructive: false, idempotent: false)`. What the opt-in changes: - The tool carries `readOnlyHint = false`, so clients such as Claude Desktop and Claude Code ask the person before each call. Beacon adds no confirmation step of its own. - It is listed to, and runs for, a signed-in user only. An app-only caller never sees it. - The audit records it as `api_request_leave:write`. - The endpoint is the authority: it validates input and applies its own business rules and record-level authorization. ## 5. Hold agents to self-service A tool call runs your endpoint as a principal with the authentication type `BeaconMcpUser`. When one endpoint serves both the browser and agents, use it to give agents less than the browser: ```csharp using Beacon.Core.Mcp; if (httpContext.User.IsBeaconMcpCall()) { // no permissions, no manager grants: the person's own records only } ``` ## 6. Make refusals reach the agent Tool calls run your `IExceptionHandler`s in registration order, as `UseExceptionHandler()` does for the browser. A deliberate refusal your application maps to a 403, 404 or 400 problem reaches the agent as that status and body. An exception no handler takes becomes the generic error `The host endpoint behind api_ failed with an unexpected error.` Set `UseHostExceptionHandlers = false` on the tool options to always answer the generic error. ## 7. Test it end to end Add `Moberg.Beacon.Testing` to the test project. In the test host, after the gateway registration, trust test tokens and register in-memory people: ```csharp var tokens = new McpTestTokens(tenantId, clientId); var users = new McpTestUsers() .Add(objectId: "oid-ada", id: "42", displayName: "Ada", new Claim("app_user_id", "42")) .Add(objectId: "oid-bob", id: "43", displayName: "Bob", new Claim("app_user_id", "43")); builder.Services.TrustMcpTestTokens(tokens); // after AddBeaconMcpGateway, in the test host only builder.Services.UseMcpTestUsers(users); await using var ada = await McpTestClient.ConnectAsync(app.GetTestClient(), tokens.For("oid-ada")); // 1. Exactly the expected tools, and the bound id is not in the schema var tools = await ada.ListToolsAsync(); tools.Select(x => x.Name).Should().BeEquivalentTo("api_my_orders", "api_request_leave", "get_context", "search", "get_documentation"); tools.Single(x => x.Name == "api_my_orders").InputSchema.GetRawText().Should().NotContain("userId"); // 2. A call returns only the caller's data var mine = await ada.CallToolAsync("api_my_orders"); mine.IsError.Should().NotBe(true); McpTestClient.Text(mine).Should().Contain("order-of-ada").And.NotContain("order-of-bob"); // 3. Naming somebody else's id is refused var other = await ada.CallToolAsync("api_my_orders", new Dictionary { ["userId"] = 43 }); other.IsError.Should().BeTrue(); McpTestClient.Text(other).Should().Contain("filled from the caller"); ``` The client is the official MCP SDK's, so the test goes through the bearer middleware, the caller mapping, the route policy, the MCP filters and the audit, and your endpoint's own authorization. Listing and calling tools needs Beacon's database (project access and the audit live there): run these tests against a real PostgreSQL or SQL Server with Beacon's schema migrated. `McpTestClient.Json(result)` returns a successful call's structured content when you want to assert on fields instead of text. Keep a pinned list of the application's tools, so adding a tool, or turning a read into a write, fails a test until someone reviews it. ## Check it works - [ ] The application starts. A missing `ReadOnly`, a bad name, an endpoint without authorization or `ReadOnly = false` without `AllowWriteTools` fails startup with a message that names the endpoint. - [ ] The test lists exactly the tools you marked, and `api_my_orders` has no `userId` argument. - [ ] A call as Ada returns Ada's data only; a call as Bob returns Bob's. - [ ] A call that sends `userId` is refused. - [ ] In Claude, the write tool asks before it runs, and the MCP audit shows it as `api_request_leave:write`. ## If something goes wrong | Symptom | Cause and fix | |---|---| | Startup names an endpoint and says `ReadOnly` was not declared | Set `ReadOnly = true` or `false`, or pass `readOnly:` to `.WithBeaconTool`. The verb is never trusted. | | Startup refuses `ReadOnly = false` | Set `AllowWriteTools = true` in the tool options. | | A Warning names a read-only endpoint on POST (or PUT, PATCH, DELETE) | Expected for a read over POST (a grid or a search with a body); it is still exposed. An endpoint that changes data is a write tool (`ReadOnly = false`). `AllowReadOnlyPost = false` exposes read-only tools on GET and HEAD only. | | A Warning says an endpoint requires only an authenticated user | Every mapped caller can call it. A self-service endpoint binds the person's id from the caller (`BindFromCaller`) and is not warned; otherwise give it a policy or roles. | | A call returns `Too many host endpoint calls are in progress` | The caller (`MaxConcurrentCallsPerCaller`, default 16) or all callers (`MaxConcurrentCalls`, default 64) stayed at their limit for the whole `QueueWait` (default 5 s). A timed-out call counts until its endpoint ends: pass `RequestAborted` / the `CancellationToken` down to your queries. | | A call says a route value does not satisfy the route constraint | The value fails the route's constraint (`{id:int}`, `regex(...)`), as it would on a real request. | | Startup refuses an endpoint with no authorization | Add `[Authorize]` or a policy, or define a fallback policy. | | Startup refuses a policy that names authentication schemes | List the scheme in `TrustedAuthenticationSchemes` only if an authorized MCP caller is an acceptable stand-in for it. Never list a step-up or MFA scheme. | | The tool is missing from the list | The caller is not mapped, cannot read the project, or the host project is not synced yet. A write tool is also hidden from system callers. | | Every call returns `Forbidden.` | The endpoint's policy denied the principal. Check the claims your resolver or principal factory sets. In full Beacon, host middleware does not run on a dispatched call: see [dispatch middleware](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/#host-middleware-and-ambient-state-dispatch-middleware). | | `... failed with an unexpected error` | The endpoint threw and no `IExceptionHandler` took it. | | A bound name is warned at startup | No tool takes that argument. Check the spelling against the endpoint's parameter. | ## Next - [Host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/): every argument source, the tool catalog above `NamedToolLimit`, dispatch and audit details. - [MCP gateway](https://moberghr.github.io/beacon/mcp/gateway/): the whole gateway recipe, including the Entra registration and `beacon-doctor`. - [Ground AI answers in your own docs and data model](https://moberghr.github.io/beacon/cookbooks/ground-with-your-docs/): give the agent documents that explain your tools. --- # Ground AI answers in your own docs and data model > Ship your Markdown documentation and your EF Core model with your application so AI agents find them through search, get_documentation and get_context, and check that an agent actually finds them. Source: https://moberghr.github.io/beacon/cookbooks/ground-with-your-docs/ | | | |---|---| | **You end up with** | Your documents imported into a Beacon project and, in full Beacon, your `DbContext` exposed as a read-only data source, plus a check that an agent finds both | | **Time** | about 25 minutes | | **You need** | An application with Beacon wired in ([MCP gateway](https://moberghr.github.io/beacon/mcp/gateway/) or [Integrate Beacon](https://moberghr.github.io/beacon/integrate/)), Markdown files in your repository, and for the data model part full Beacon with an EF Core `DbContext` and a read-only database login | | **Applies to** | Beacon 5.0 · Both. Documents: gateway and full Beacon. `ExposeDbContext`, the glossary and golden examples: full Beacon only, because the gateway has no data sources and no SQL tools | :::tip[Let your AI do it] Paste this into Claude Code (or another coding agent) in your application's repository. ```text wrap Read https://moberghr.github.io/beacon/cookbooks/ground-with-your-docs.md and follow it step by step in this repository. Ask me whether this is the MCP gateway or full Beacon, which folder holds the docs, and which tables may be exposed. Never expose a table I did not name, never put a connection string in code, and show me the result of each "Check it works" item. ``` ::: ## Before you start | Decision | Options | Default | |---|---|---| | Where the documents live | A folder copied to the output, or embedded in the assembly | Embedded, so the deployment cannot lose them | | Expose the data model | Yes (full Beacon only), or documents only | Documents only | | Tables an agent may read | An explicit allow-list | None: `ExposeDbContext` is default-deny | | Columns hidden or masked | Excluded (invisible) or masked (visible, values `***`) | Mask national ids and account numbers, exclude secrets | ## 1. Write documents that ground well Each document is split into overlapping sentence windows (5 sentences, 1 overlap, by default), so a paragraph is retrieved on its own. Write for that. - [ ] One topic per page, with a `title:` in the frontmatter (else the first `# ` heading is used). - [ ] Each paragraph makes sense without the paragraphs around it: name the thing in full instead of "it". - [ ] Use the words your people use in questions, and say once what they mean: "active customer means ...". - [ ] Put definitions, rules and "where to find it" first, narrative last. - [ ] State units, time zones and currencies. - [ ] Keep each file under 512 KB (the default `MaxDocumentBytes`; larger files are skipped with a warning). - [ ] No secrets, no personal data, no real customer names. The documents are readable by every caller who can read the project. Only `.md`, `.markdown` and `.txt` files are imported. ## 2. Ship the documents Embed them in the assembly: ```xml ``` The `LogicalName` is required: without it MSBuild mangles folder names and joins them with dots. To copy a folder instead, see [Shipping the documents](https://moberghr.github.io/beacon/mcp/host-docs/#shipping-the-documents). ## 3. Register the documents In the gateway: ```csharp builder.Services.AddBeaconMcpGateway(builder.Configuration, x => { // ... x.Docs(y => y.FromEmbeddedResources(typeof(Program).Assembly, "docs/wiki/")); }); ``` The gateway sets the project name for you and syncs the host project the first time it is needed. You add no startup call. In full Beacon: ```csharp var beacon = builder.Services.AddBeaconServices(builder.Configuration, o => { /* your options */ }); beacon.ExposeDocs(o => { o.ProjectName = "Back Office"; // the project every Expose* call of this host uses o.FromEmbeddedResources(typeof(Program).Assembly, "docs/wiki/"); // or o.FromDirectory("docs/wiki").Exclude("drafts/**"); }); beacon.UseSqlServer(); // ends the builder chain: call ExposeDocs before it var app = builder.Build(); // ... apply migrations ... await app.Services.SyncBeaconHostAsync(); // data sources first, then documents ``` `SyncBeaconHostAsync` is idempotent: a document whose content hash is unchanged is left alone, a changed one is updated and re-chunked, a removed file is archived. It is safe to run from several replicas at once. ## 4. Expose the data model (full Beacon only) Skip this step in the gateway: it has no data sources, so there is nothing for `ExposeDbContext` to attach to and no SQL tool to ground. ```csharp beacon.ExposeDbContext(o => { o.Name = "Shop"; o.ProjectName = "Back Office"; // the same project as ExposeDocs o.ReadOnlyConnection("BeaconReadOnly") // ConnectionStrings:BeaconReadOnly, a read-only login .AllowTables("Loan", "LoanInstallment", "Customer") .ExcludeColumns(x => x.Table == "Customer" && x.Name.StartsWith("Kyc")) .MaskColumns(x => x.Name is "SSN" or "AccountNumber"); }); ``` Beacon reads the tables, columns, keys and foreign keys from the EF model without opening a connection, and takes descriptions from `HasComment` or, with `GenerateDocumentationFile` on, from your XML `` comments. Write those comments in business words: they become the column and table descriptions the agent reads. Columns whose names contain `password`, `secret`, `token` and similar words are excluded whatever you write. See [Host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/) for every option. ## 5. How it reaches agents | Tool | What the agent gets from your material | |---|---| | `search` | Documents as `[IMPORTED_DOC]` hits with their path and title, found by keyword, and by meaning when embeddings are enabled. Tables and columns too, in full Beacon. | | `get_documentation` | The project-level answer ends with a list of your documents. `document` takes a path or an exact title and returns that document in full. `datasource_name`, `table_name` and `schema_name` narrow to the data model. | | `get_context` | `format: "agents_md"` returns a project brief: data sources, key tables, glossary terms, document titles and how to use the tools. Built from the catalog, with no LLM call. | | `ask` (full Beacon) | A conceptual question is answered from project documentation, including your documents; a data question is grounded in the schema, sample values, glossary and verified examples. | | `get_query_context` (full Beacon) | The same grounding context `ask` would use, so an agent can write its own SQL. | Documents work with no LLM and no embedder: keyword search matches title, path and chunk text. Embeddings, when configured, add meaning-based matching ([Semantic retrieval](https://moberghr.github.io/beacon/mcp/knowledge-base/#semantic-retrieval)). ## 6. Teach the vocabulary and verified answers (full Beacon only) Two kinds of grounding are curated by an Admin after the data source exists: - **Business glossary**: terms, synonyms and a metric expression, through `/beacon/api/glossary` (API only, no page). The top matches (`GlossaryTopK`, default 5) enter the prompt. See [Business glossary](https://moberghr.github.io/beacon/mcp/knowledge-base/#business-glossary). - **Golden examples**: human-verified question to SQL pairs. An Admin promotes an answer rated `correct` on the **MCP Learning** page (`/beacon/mcp-learning`, tab **Golden candidates**). The top matches (`GoldenExemplarTopK`, default 4) enter the prompt as authoritative. See [Golden examples](https://moberghr.github.io/beacon/mcp/knowledge-base/#golden-examples-and-the-eval-harness). Both are scoped to the project, so another project that shares the data source never sees them. ## Check it works - [ ] Startup completes. In full Beacon `SyncBeaconHostAsync` ran after migrations. - [ ] `GET /beacon/api/projects/{id}/imported-documents` returns your documents (needs Read in the project). - [ ] Connect an MCP client ([Connect Claude Code, Cursor or VS Code](https://moberghr.github.io/beacon/cookbooks/connect-coding-agent/)) and say this: ```text wrap title="Say to Claude" Search the Back Office project for "active customer" and show me which documents match. Then read the best match in full with get_documentation. ``` - [ ] `search` answers with an `[IMPORTED_DOC]` line for your page, and `get_documentation` with `document` set to its path returns the text. - [ ] Full Beacon only: `get_documentation` with `table_name` returns the descriptions you wrote, and masked columns are flagged as masked. - [ ] Change one sentence in a document, redeploy, and the new text comes back from `get_documentation`. ## If something goes wrong | Symptom | Cause and fix | |---|---| | No documents are listed | The sync has not run (full Beacon: call `SyncBeaconHostAsync` after migrations), the folder is wrong, or the glob matched nothing. A configured directory that does not exist fails startup. | | Folder names look like `_40_web` or the path has dots | The embedded resources have no `LogicalName`. Set it as in step 2. | | A file is missing and the log has a size warning | It is over `MaxDocumentBytes` (512 KB). Split it. | | Two files with names that differ only by case, one missing | Paths compare ignoring case; the second is skipped with a warning naming its path. Rename one. | | The project is called `Back Office (host)` | A project that is not host-managed already had that name. The host project is created beside it. See [The host project](https://moberghr.github.io/beacon/mcp/host-docs/#the-host-project). | | `search` finds nothing for a paraphrase | Embeddings are off, so matching is by keyword. Use the people's own words in the document or enable embeddings. | | A table is not visible to the agent | It is not in `AllowTables` / `AllowEntity`, or the caller cannot read the project. | | A query on a host data source is refused for an unqualified table | Host data source SQL must be a single `SELECT` with schema-qualified table names, such as `dbo.Customer`. | ## Next - [Host docs](https://moberghr.github.io/beacon/mcp/host-docs/): every option, the sync rules and the REST routes for imported documents. - [Host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/): the allow-list, masking and the execution-time policy. - [Knowledge base and SQL grounding](https://moberghr.github.io/beacon/mcp/knowledge-base/): what goes into an `ask` prompt and who curates it. - [Ask your data questions](https://moberghr.github.io/beacon/cookbooks/ask-your-data/): use the grounding you just built. --- # Embed Beacon in an existing ASP.NET Core app (PostgreSQL + Warp) > Beacon's UI, API and MCP server running inside your ASP.NET Core application, storing its data in PostgreSQL and running its jobs on Moberg Warp, with a first admin and one monitored data source. Source: https://moberghr.github.io/beacon/cookbooks/embed-postgresql-warp/ | | | |---|---| | **You end up with** | Your ASP.NET Core app serving Beacon at `/beacon` (UI, REST API, MCP server), Beacon's schema created in PostgreSQL, scheduled jobs running on Warp, a first admin, and one data source added with a read-only login. Your own routes stay as they are. | | **Time** | about 45 minutes | | **You need** | An ASP.NET Core app on .NET 10, a PostgreSQL server with the [pgvector](https://github.com/pgvector/pgvector) extension available and an empty database for Beacon, and the right to create a read-only login on the database you want to monitor. Read [Integrate Beacon](https://moberghr.github.io/beacon/integrate/) first if you have not. | | **Applies to** | Beacon 5.0 · Full Beacon | :::tip[Let your AI do it] Paste this into Claude Code or another coding agent, in the repository of your app. ```text wrap Read https://moberghr.github.io/beacon/cookbooks/embed-postgresql-warp.md and follow it step by step in this repository. Ask me before every decision it marks as one, and show me the result of each "Check it works" item. Never write the connection string, the encryption key or the setup token into a file that is committed: use user-secrets or environment variables. Do not run the app against a database I have not named. ``` ::: ## Before you start | Decision | Options | Default | |---|---|---| | Job runner | Moberg Warp (this recipe), or another runner: see [Embed Beacon on SQL Server with Hangfire](https://moberghr.github.io/beacon/cookbooks/embed-sqlserver-hangfire/) | Warp | | Warp's database | The Beacon database, or a separate PostgreSQL database (`ConnectionStrings:WarpContext`) | The Beacon database | | Who signs in | Beacon's own user store (this recipe), or your app's sign-in: see [Sign-in & identity](https://moberghr.github.io/beacon/integrate/authentication/#your-app-owns-sign-in) | Beacon's user store | | Connectors | One per engine you monitor; this recipe adds PostgreSQL | PostgreSQL | ## 1. Add the packages ```bash dotnet add package Moberg.Beacon.Core.PostgreSql dotnet add package Moberg.Beacon.Api dotnet add package Moberg.Beacon.UI dotnet add package Moberg.Beacon.AI dotnet add package Moberg.Beacon.MCP dotnet add package Moberg.Beacon.Connector.PostgreSql dotnet add package Moberg.Warp.Core --version 3.11.0 dotnet add package Moberg.Warp.Worker --version 3.11.0 dotnet add package Moberg.Warp.Provider.PostgreSql --version 3.11.0 dotnet add package EFCore.NamingConventions dotnet add package Microsoft.EntityFrameworkCore.Design ``` `Moberg.Beacon.Core` comes with the others. Keep every `Moberg.Beacon.*` package on one version ([Packages](https://moberghr.github.io/beacon/integrate/packages/#versions)). Drop `Moberg.Beacon.AI` and `Moberg.Beacon.MCP` if you want neither; [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/#minimal-variants) lists what else goes. ## 2. Set the two secrets ```bash dotnet user-secrets init dotnet user-secrets set "ConnectionStrings:BeaconContext" "Host=localhost;Database=beacon;Username=beacon;Password=…" dotnet user-secrets set "Beacon:EncryptionKey" "$(openssl rand -base64 32)" ``` Create the database first (`CREATE DATABASE beacon;`). Beacon creates the schema and tables in it, not the database. Outside development the same two keys come from your secret store or from `ConnectionStrings__BeaconContext` and `Beacon__EncryptionKey`. Beacon refuses to start with a missing, short or non-random key. Keep the key: stored connection strings cannot be read without it. ## 3. Write the Warp side Beacon runs no jobs itself. Your host gives it an `IBeaconScheduler`, and each job marks its scope as system work before Beacon code runs. An unmarked job scope gets no project access and logs a warning ([the system-work rule](https://moberghr.github.io/beacon/integrate/background-jobs/#the-system-work-rule)). `WarpDbContext.cs` holds Warp's own tables (the `warp` schema), apart from Beacon's: ```csharp using Microsoft.EntityFrameworkCore; public sealed class WarpDbContext(DbContextOptions options) : DbContext(options); ``` `BeaconWarp.cs` registers Warp, applies its migrations and registers the recurring maintenance jobs: ```csharp using Beacon.Core.Authorization; using Beacon.Core.Worker; using Microsoft.EntityFrameworkCore; using Warp.Core; using Warp.Core.Data.Entities; using Warp.Core.Handlers; using Warp.Provider.PostgreSql; using Warp.Worker; public static class BeaconWarp { public static IServiceCollection AddBeaconWarp(this IServiceCollection services, IConfiguration configuration) { services.AddDbContext(x => x .UseNpgsql(configuration.GetConnectionString("WarpContext") ?? configuration.GetConnectionString("BeaconContext")) .UseSnakeCaseNamingConvention()); services.AddWarpServer(x => { x.UsePostgreSql(); x.PollingInterval = TimeSpan.FromSeconds(1); x.WorkerCount = Math.Min(Environment.ProcessorCount * 2, 20); // protects the connection pool // No x.AddRetry(): a failed check is investigated, not retried and notified twice. }); return services; } public static async Task StartBeaconWarpAsync(this WebApplication app) { using var scope = app.Services.CreateScope(); scope.ServiceProvider.GetRequiredService().Database.Migrate(); var recurring = scope.ServiceProvider.GetRequiredService(); await recurring.AddOrUpdateRecurringJob(new CleanupOldSignalsJob(), "mcp-learning-cleanup", "0 3 * * *"); // ... one line per job in the table below ... } } // Warp registers this behaviour for every job. It is what marks the job's scope as system work. public sealed class SystemWorkBehavior(ISystemWorkScope systemWork) : IPipelineBehavior where TRequest : IRequest { public Task HandleAsync(TRequest request, RequestHandlerDelegate next, CancellationToken cancellationToken) { systemWork.MarkAsSystemWork(); return next(request, cancellationToken); } } public sealed class ExecuteSubscriptionQueryJob : IJob { public int SubscriptionId { get; init; } } public sealed class ExecuteSubscriptionQueryJobHandler(IJobService jobs) : IJobHandler { public Task HandleAsync(ExecuteSubscriptionQueryJob message, CancellationToken ct) => jobs.ExecuteQuery(message.SubscriptionId, ct); } public sealed class EvaluateDataContractJob : IJob { public int ContractId { get; init; } } public sealed class EvaluateDataContractJobHandler(IJobService jobs) : IJobHandler { public Task HandleAsync(EvaluateDataContractJob message, CancellationToken ct) => jobs.EvaluateDataContract(message.ContractId, ct); } public sealed class CleanupOldSignalsJob : IJob; public sealed class CleanupOldSignalsJobHandler(IJobService jobs) : IJobHandler { public Task HandleAsync(CleanupOldSignalsJob message, CancellationToken ct) => jobs.CleanupOldSignals(ct); } public sealed class BeaconScheduler(IRecurringJobPublisher recurring, WarpDbContext warp) : IBeaconScheduler { public Task AddOrUpdate(int subscriptionId, string subscriptionName, string cron) => recurring.AddOrUpdateRecurringJob(new ExecuteSubscriptionQueryJob { SubscriptionId = subscriptionId }, $"subscription-{subscriptionId}", cron); public Task Remove(int subscriptionId, string subscriptionName) => RemoveRecurring($"subscription-{subscriptionId}"); public Task AddOrUpdateDataQualityJob(int contractId, string contractName, string cron) => recurring.AddOrUpdateRecurringJob(new EvaluateDataContractJob { ContractId = contractId }, $"dq-{contractId}", cron); public Task RemoveDataQualityJob(int contractId, string contractName) => RemoveRecurring($"dq-{contractId}"); private async Task RemoveRecurring(string name) { var job = await warp.Set().Where(x => x.Name == name).FirstOrDefaultAsync(); if (job != null) { warp.Set().Remove(job); await warp.SaveChangesAsync(); } } } ``` Recurring jobs are keyed by the entity id, never by the name. Five of the other `IBeaconScheduler` members (`AddOrUpdateDataQualityJob`, `RemoveDataQualityJob`, `EnqueueProjectDocumentation`, `EnqueueAiActorThinkCycle`, `EnqueueMcpEval`) have default bodies that throw `NotImplementedException` until you implement them, and `EnqueueMetadataRefresh` and `EnqueueBackgroundTask` default to doing nothing (a background task then runs in the request that started it). Implement the ones for the features you use. Beacon also expects these recurring maintenance jobs. `mcp-learning-cleanup` is in the code above. Write one `IJob` plus handler for each of the others, in the same shape, and register each in `StartBeaconWarpAsync` with `recurring.AddOrUpdateRecurringJob(new XJob(), id, cron)` (`recurring` is the scope's `IRecurringJobPublisher`). The handler code is in [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/#reference-implementation-moberg-warp). | Id | Cron | Handler calls | |---|---|---| | `mcp-learning-aggregate` | `0 */6 * * *` | `IJobService.AggregateLearnedPatterns` | | `mcp-learning-cleanup` | `0 3 * * *` | `IJobService.CleanupOldSignals` | | `mcp-audit-retention` | `30 3 * * *` | `IMcpAuditRetentionService.PurgeExpiredAsync` | | `security-audit-retention` | `45 3 * * *` | `ISecurityAuditRetentionService.PurgeExpiredAsync` | | `security-audit-verify` | `15 * * * *` | `ISecurityAuditVerificationService.VerifyNewEventsAsync` | | `mcp-embedding-reindex` | `0 */12 * * *` | `IJobService.ReindexEmbeddings` | | `mcp-docchunk-reindex` | `0 */12 * * *` | `IJobService.ReindexDocChunks` | Create Warp's initial migration once: `dotnet ef migrations add WarpInitial --context WarpDbContext`. ## 4. Register Beacon and map its endpoints `BeaconClaimsTransformation` is a small class you copy from [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/#the-claims-transformation): it adds the role claim from Beacon's user store. Then `Program.cs`: ```csharp using Beacon.AI; using Beacon.Api; using Beacon.Api.Authentication; using Beacon.Api.Endpoints; using Beacon.Connector.PostgreSql; using Beacon.Core; using Beacon.Core.Authentication.Providers; using Beacon.Core.PostgreSql; using Beacon.MCP; using Beacon.MCP.Discovery; using Beacon.UI; var builder = WebApplication.CreateBuilder(args); const string beaconBasePath = BeaconUiEndpointRouteBuilderExtensions.BasePath; // "/beacon" // ... your own services stay here ... builder.Services.AddBeaconWarp(builder.Configuration); builder.Services.AddBeaconHostInfrastructure(); builder.Services.AddBeaconServices(builder.Configuration, options => { options.AddBeaconScheduler(); options.BaseUrl = builder.Configuration["Beacon:BaseUrl"]; // public URL including /beacon, for links in notifications options.UseAI = true; options.Authorization.Enabled = true; options.Authentication.EnableLoginForm = true; options.AddAuthenticationProvider(); options.EnableUserManagement(); }) .AddPostgreSqlConnector() .UsePostgreSql(); // reads ConnectionStrings:BeaconContext; must end the chain builder.Services.AddBeaconCookieAuthentication(beaconBasePath); builder.Services.AddBeaconDataProtection(builder.Configuration, builder.Environment); builder.Services.AddBeaconOidcAuthentication(builder.Configuration); // does nothing unless Oidc:Enabled builder.Services.AddBeaconAI(builder.Configuration); builder.Services.AddBeaconMcp(); builder.Services.AddAntiforgery(x => { x.HeaderName = "X-XSRF-TOKEN"; x.Cookie.Name = ".Beacon.Antiforgery"; }); builder.Services.AddBeaconApiAuthorization(); builder.Services.AddBeaconApiServices(); var app = builder.Build(); var beaconConfiguration = app.Services.GetRequiredService(); Beacon.Core.ServiceConfiguration.UseBeacon(app.Services, createSchema: true); // The order is load-bearing. app.UseHttpsRedirection(); app.UseStaticFiles(); app.UseMcpResourceMetadataChallenge(); app.UseBeaconAuthentication(); // API key, bearer JWT, UseAuthentication, Beacon.Auth cookie app.UseAuthorization(); app.UseMiddleware(beaconConfiguration, beaconBasePath); app.UseMiddleware(beaconConfiguration, beaconBasePath); app.UseAntiforgery(); app.MapBeaconApi(); app.MapLoginEndpoints(beaconBasePath, beaconConfiguration); app.MapSetupEndpoints(beaconBasePath); app.MapBeaconMcp(); app.MapMcpDiscovery(); app.MapBeaconUi(); // ... your own endpoints stay here ... await app.StartBeaconWarpAsync(); app.Run(); ``` What to know about this file: - `UseBeacon(app.Services, createSchema: true)` is the schema step. It creates the `beacon` schema if missing and applies the pending migrations, synchronously, before the host serves a request. One migration runs `CREATE EXTENSION IF NOT EXISTS vector;`, so the login needs the right to create it, or an administrator installs it first ([pgvector](https://moberghr.github.io/beacon/integrate/database/#pgvector-on-postgresql)). - With several instances, run migrations once from a deployment step instead: `await Beacon.Core.PostgreSql.BeaconPostgreSqlMigrator.MigrateAsync(connectionString);` in a migrator process that does not compose Beacon ([Metadata database](https://moberghr.github.io/beacon/integrate/database/#creating-and-migrating-the-schema)). - Beacon's middlewares act on `/beacon` only. Your routes keep your authentication. - Behind a proxy, add `UseForwardedHeaders` first ([Production checklist](https://moberghr.github.io/beacon/integrate/production/)). ## 5. Create the first admin Start the app. While no admin exists, Beacon prints a one-time setup token to the process console (standard error, not the log): ```text wrap Beacon first-run setup token (enter it on the setup page): 3f1c… ``` Open `https://localhost:/beacon/setup`, enter the token, then a user name and password (at least 8 characters, with upper case, lower case, a digit and a special character), and click **Create super admin**. With several replicas set `Beacon:UserManagement:SetupToken` (at least 32 characters) from your secret store, because each replica generates its own. Details: [First-run setup](https://moberghr.github.io/beacon/security/user-management/#first-run-setup). ## 6. Add a data source with a read-only login On the server you want to monitor, create a login that cannot write, for PostgreSQL: ```sql CREATE ROLE beacon_reader LOGIN PASSWORD '…'; ALTER ROLE beacon_reader SET default_transaction_read_only = on; GRANT USAGE ON SCHEMA public TO beacon_reader; GRANT SELECT ON ALL TABLES IN SCHEMA public TO beacon_reader; ``` In Beacon, open **Data Sources**, click **New data source**, pick **Database**, choose the **Database engine**, paste the **Connection string**, click **Test connection** (wait for **Connected**), then **Create data source**. Outside development Beacon refuses a connection that does not verify the server's certificate, for PostgreSQL `SSL Mode=VerifyFull` ([Transport security](https://moberghr.github.io/beacon/features/data-sources/#transport-security-tls)). ## Check it works - [ ] The app starts without `Implementation of IBeaconScheduler is required.` or an encryption-key error. - [ ] `psql` shows the `beacon` schema with tables, and `\dx` shows `vector`. - [ ] `https://localhost:/beacon/` redirects to `/beacon/setup`, and after step 5 to `/beacon/login`. Your own routes still answer as before. - [ ] `curl -k https://localhost:/beacon/api/health` returns `{"status":"ok"}`, also before the first admin exists. A `503` means Beacon cannot reach its database. - [ ] The data source shows **Connected** and its tables appear after the metadata load. - [ ] The `warp` schema exists and `warp.recurring_job` lists the maintenance jobs you registered. - [ ] After you create a subscription, `warp.recurring_job` lists `subscription-`. ## If something goes wrong | Symptom | Cause and fix | |---|---| | `Implementation of IBeaconScheduler is required.` | `options.AddBeaconScheduler()` is missing from the `AddBeaconServices` lambda. | | Startup refuses the encryption key | It must be base64 of at least 32 random bytes: `openssl rand -base64 32`. A `${…}` placeholder that was never expanded fails too. | | `extension "vector" is not available` | Install pgvector on the server, or on Azure Database for PostgreSQL add `VECTOR` to `azure.extensions`. | | Startup fails outside Development about Data Protection | Set `Beacon:DataProtection:ApplicationName` and chain `ProtectKeysWithCertificate(...)` or `ProtectKeysWithAzureKeyVault(...)`, or configure your own Data Protection and drop `AddBeaconDataProtection` ([Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/#data-protection)). | | Warning that a scope is not marked as system work, and jobs do nothing | A job ran without `ISystemWorkScope.MarkAsSystemWork()`. Check that `SystemWorkBehavior` is compiled into your host project, which is where Warp's source generator registers it. | | `CS9137` when you add `AddOpenApi()` | Reference `Microsoft.AspNetCore.OpenApi` directly ([Packages](https://moberghr.github.io/beacon/integrate/packages/#build-time-notes)). | | The data source is refused for its connection string | The connection does not verify the server over TLS. Fix the connection string; do not look for a way to turn the check off. | ## Next - [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/): every option of `AddBeaconServices`, the middleware table and the routes. - [Sign in to Beacon with Microsoft Entra ID](https://moberghr.github.io/beacon/cookbooks/entra-sso/): single sign-on for the UI. - [Production checklist](https://moberghr.github.io/beacon/integrate/production/): proxy, TLS, key rotation, backups. - [MCP gateway for your application](https://moberghr.github.io/beacon/mcp/gateway/): if you want AI clients on your own endpoints, not on data sources. --- # Embed Beacon on SQL Server with Hangfire > Beacon inside your ASP.NET Core app with its own data in SQL Server and its scheduled jobs on Hangfire: what differs from the PostgreSQL and Warp recipe, and the IBeaconScheduler you write for Hangfire. Source: https://moberghr.github.io/beacon/cookbooks/embed-sqlserver-hangfire/ | | | |---|---| | **You end up with** | Your ASP.NET Core app serving Beacon at `/beacon`, Beacon's schema created in SQL Server, and subscriptions, data-quality checks and maintenance running as Hangfire jobs. | | **Time** | about 40 minutes | | **You need** | An ASP.NET Core app on .NET 10, the [PostgreSQL and Warp recipe](https://moberghr.github.io/beacon/cookbooks/embed-postgresql-warp/) open beside this one, an empty SQL Server database for Beacon, and a login that may create a schema in it. | | **Applies to** | Beacon 5.0 · Full Beacon | :::tip[Let your AI do it] Paste this into Claude Code or another coding agent, in the repository of your app. ```text wrap Read https://moberghr.github.io/beacon/cookbooks/embed-sqlserver-hangfire.md and follow it step by step in this repository. It lists only what differs from https://moberghr.github.io/beacon/cookbooks/embed-postgresql-warp.md, so read both. Ask me before every decision either one marks as one, and show me the result of each "Check it works" item. Never write a connection string, the encryption key or the setup token into a committed file. Tell me clearly that the Hangfire scheduler is code we write in our host: Beacon does not ship one. ``` ::: This recipe lists only the differences from [the PostgreSQL and Warp recipe](https://moberghr.github.io/beacon/cookbooks/embed-postgresql-warp/). Follow that recipe for the secrets, the middleware order, first-run setup and the first data source, and swap in what is below. ## Before you start | Decision | Options | Default | |---|---|---| | Hangfire's storage | The Beacon database (Hangfire keeps its tables in its own `HangFire` schema), or a separate SQL Server database | A separate database if you can; the Beacon database otherwise | | Which data sources you monitor | Add the matching connector for each engine. `Moberg.Beacon.Connector.SqlServer` for SQL Server | The engines you actually monitor | | Several app instances | Run Beacon's migrations once from a deployment step, not from every replica | A deployment step | ## 1. Swap the packages ```bash dotnet add package Moberg.Beacon.Core.SqlServer dotnet add package Moberg.Beacon.Api dotnet add package Moberg.Beacon.UI dotnet add package Moberg.Beacon.AI dotnet add package Moberg.Beacon.MCP dotnet add package Moberg.Beacon.Connector.SqlServer dotnet add package Hangfire.AspNetCore dotnet add package Hangfire.SqlServer ``` `Moberg.Beacon.Core.SqlServer` stores **Beacon's own data** in SQL Server. `Moberg.Beacon.Connector.SqlServer` lets Beacon **monitor** a SQL Server database. Add the connector only if you monitor one. The packages `Moberg.Warp.*`, `EFCore.NamingConventions` and the `Warp*` files are not needed. ## 2. Change the registration In the `AddBeaconServices` chain, replace the PostgreSQL calls: ```csharp using Beacon.Connector.SqlServer; // AddSqlServerConnector using Beacon.Core.SqlServer; // UseSqlServer builder.Services.AddBeaconServices(builder.Configuration, options => { options.AddBeaconScheduler(); // the class from step 3 // ... the rest of the options as in the PostgreSQL recipe ... }) .AddSqlServerConnector() .UseSqlServer(); // reads ConnectionStrings:BeaconContext; must end the chain ``` `ConnectionStrings:BeaconContext` is now a SQL Server connection string. Everything lives in the fixed schema `beacon`; `Beacon:Schema` accepts nothing else. Replace `AddBeaconWarp(...)` and `StartBeaconWarpAsync()` with Hangfire's registration and the startup of step 4: ```csharp builder.Services.AddHangfire(x => x.UseSqlServerStorage(builder.Configuration.GetConnectionString("HangfireContext") ?? builder.Configuration.GetConnectionString("BeaconContext"))); builder.Services.AddHangfireServer(); ``` Keep `AddBeaconHostInfrastructure<...>()`, the cookie, Data Protection, API, MCP and UI calls and the whole middleware pipeline exactly as in the other recipe. Data Protection keeps its keys in Beacon's database on either provider. ## 3. Write the Hangfire scheduler Beacon ships no job runner and no Hangfire integration. `IBeaconScheduler` is the contract between Beacon and your runner, and this class is **your code**, in your host. The members, from `src/Beacon.Core/Worker/IBeaconScheduler.cs`: | Member | Needed for | |---|---| | `AddOrUpdate(subscriptionId, subscriptionName, cron)` and `Remove(subscriptionId, subscriptionName)` | Subscriptions. Required: they have no default body. | | `AddOrUpdateDataQualityJob(contractId, contractName, cron)` and `RemoveDataQualityJob(contractId, contractName)` | Data contracts. Throw `NotImplementedException` until you implement them. | | `EnqueueProjectDocumentation(projectId, userId, notifyUserId)` | AI documentation. Returns your runner's job id. Throws until implemented. | | `EnqueueAiActorThinkCycle(actorId, subscriptionId)` | AI actors. Returns a job id. Throws until implemented. | | `EnqueueMcpEval(runId)` | MCP evaluation runs. Returns a job id. Throws until implemented. | | `EnqueueMetadataRefresh(dataSourceId)` | Re-scan after a data source was pointed at another target. Optional: the default returns `null` and Beacon logs that the old schema stays until a refresh. | | `EnqueueBackgroundTask(taskId, notifyUserId)` | A task a user started and follows (a background metadata refresh); your job calls `IBackgroundTaskService.ExecuteAsync(taskId, ct)`. Optional: the default returns `null` and Beacon runs the task in the request. | `cron` is a standard 5-field expression, already validated by Beacon. Key recurring jobs by the entity id, never by the name. Every job opens its own DI scope, marks it as system work with `ISystemWorkScope.MarkAsSystemWork()` **before** it calls Beacon, and calls one `IJobService` method. A scope that was not marked gets no project access and logs a warning. Work done for a user, such as a subscription's run, is still executed as that user: Beacon sets the owner itself ([the system-work rule](https://moberghr.github.io/beacon/integrate/background-jobs/#the-system-work-rule)). Do not resolve the current user in your wrapper. ```csharp using Beacon.Core.Authorization; using Beacon.Core.Worker; using Hangfire; [AutomaticRetry(Attempts = 0)] // a failed check is investigated; a retry would notify twice public sealed class BeaconHangfireJobs(ISystemWorkScope systemWork, IJobService jobs) { public Task ExecuteSubscription(int subscriptionId, CancellationToken ct) { systemWork.MarkAsSystemWork(); return jobs.ExecuteQuery(subscriptionId, ct); } public Task EvaluateContract(int contractId, CancellationToken ct) { systemWork.MarkAsSystemWork(); return jobs.EvaluateDataContract(contractId, ct); } public Task RunMcpEval(int runId, CancellationToken ct) { systemWork.MarkAsSystemWork(); return jobs.RunMcpEval(runId, ct); } public Task CleanupOldSignals(CancellationToken ct) { systemWork.MarkAsSystemWork(); return jobs.CleanupOldSignals(ct); } // AggregateLearnedPatterns, ReindexEmbeddings and ReindexDocChunks follow the same shape. } public sealed class HangfireBeaconScheduler(IRecurringJobManager recurring, IBackgroundJobClient background) : IBeaconScheduler { public Task AddOrUpdate(int subscriptionId, string subscriptionName, string cron) { recurring.AddOrUpdate($"subscription-{subscriptionId}", x => x.ExecuteSubscription(subscriptionId, CancellationToken.None), cron); return Task.CompletedTask; } public Task Remove(int subscriptionId, string subscriptionName) { recurring.RemoveIfExists($"subscription-{subscriptionId}"); return Task.CompletedTask; } public Task AddOrUpdateDataQualityJob(int contractId, string contractName, string cron) { recurring.AddOrUpdate($"dq-{contractId}", x => x.EvaluateContract(contractId, CancellationToken.None), cron); return Task.CompletedTask; } public Task RemoveDataQualityJob(int contractId, string contractName) { recurring.RemoveIfExists($"dq-{contractId}"); return Task.CompletedTask; } public Task EnqueueMcpEval(int runId) => Task.FromResult(background.Enqueue(x => x.RunMcpEval(runId, CancellationToken.None))); } ``` Hangfire's ASP.NET Core integration creates one DI scope per job and builds `BeaconHangfireJobs` in it, so the `ISystemWorkScope` the wrapper marks is the one `IJobService` reads. Hangfire fills the `CancellationToken` argument with its shutdown token at run time. The Hangfire calls above (`AddOrUpdate`, `RemoveIfExists`, `Enqueue`, `[AutomaticRetry]`) come from Hangfire, not from Beacon: check them against the Hangfire version you install. For AI documentation and AI actors, add wrapper methods that call `IProjectDocumentationService.GenerateDocumentationAsync(projectId, userId, ct)` and `AiActorService.ExecuteThinkCycleBackgroundAsync(actorId, subscriptionId)` ([Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/#reference-implementation-moberg-warp) names their namespaces). ## 4. Register the maintenance jobs Beacon expects these recurring jobs. Register each at startup, after `app.Build()`, with `IRecurringJobManager`: | Id | Cron | Calls | |---|---|---| | `mcp-learning-aggregate` | `0 */6 * * *` | `IJobService.AggregateLearnedPatterns` | | `mcp-learning-cleanup` | `0 3 * * *` | `IJobService.CleanupOldSignals` | | `mcp-audit-retention` | `30 3 * * *` | `IMcpAuditRetentionService.PurgeExpiredAsync` | | `security-audit-retention` | `45 3 * * *` | `ISecurityAuditRetentionService.PurgeExpiredAsync` | | `security-audit-verify` | `15 * * * *` | `ISecurityAuditVerificationService.VerifyNewEventsAsync` | | `mcp-embedding-reindex` | `0 */12 * * *` | `IJobService.ReindexEmbeddings` | | `mcp-docchunk-reindex` | `0 */12 * * *` | `IJobService.ReindexDocChunks` | ```csharp var recurring = app.Services.GetRequiredService(); recurring.AddOrUpdate("mcp-learning-cleanup", x => x.CleanupOldSignals(CancellationToken.None), "0 3 * * *"); // ... one line per row of the table, each with its own wrapper method ... ``` While `Beacon:PreviousEncryptionKeys` lists a retired key, also run `IStoredValueReEncryptionService.ReEncryptAsync` once per start ([Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/#recurring-maintenance-jobs)). The Warp dashboard and the live `JobStatusChanged` progress push are features of the reference host's Warp setup; with Hangfire the UI still works, it just shows no live progress for documentation jobs. ## 5. Create the schema in SQL Server Beacon does not migrate itself. For one instance, call this after `builder.Build()`, as in the other recipe: ```csharp Beacon.Core.ServiceConfiguration.UseBeacon(app.Services, createSchema: true); ``` It creates the `beacon` schema when it is missing and applies the pending migrations. The login needs the right to create the schema and the tables in it. If it cannot, create the `beacon` schema as an administrator and call `UseBeacon(app.Services)` without `createSchema`. With several instances, or a separate migrator process that does not compose Beacon, call the standalone migrator once from a deployment step: ```csharp await Beacon.Core.SqlServer.BeaconSqlServerMigrator.MigrateAsync(connectionString); ``` It creates the schema and applies the migrations, and it is idempotent. Both paths write the history table `beacon.__EFMigrationsHistory`. SQL Server needs no extension. ## 6. Know what SQL Server does without pgvector On PostgreSQL, Beacon keeps embeddings in a `vector(384)` column with an HNSW index. On SQL Server there is no such column: Beacon stores the embedding bytes and computes similarity over them in memory. The embedding and document re-index jobs still run. As on PostgreSQL, retrieval is semantic only when you add `Moberg.Beacon.AI.Embeddings.Onnx` and call `AddBeaconOnnxEmbeddings()`; without it retrieval stays keyword-based. With it, SQL Server has no vector index to lean on, so the similarity work grows with the number of indexed items. ## 7. Monitoring SQL Server: the read-only login For a data source on SQL Server, create a login that can only read. Beacon's SQL Server path relies on its SQL checks and this login, because SQL Server has no read-only session ([Data sources](https://moberghr.github.io/beacon/features/data-sources/)): ```sql CREATE LOGIN beacon_reader WITH PASSWORD = '…'; USE YourDatabase; CREATE USER beacon_reader FOR LOGIN beacon_reader; ALTER ROLE db_datareader ADD MEMBER beacon_reader; ``` The connection string must verify the server: `Encrypt` Mandatory or Strict and no `TrustServerCertificate`, for example `Server=db.example.com;Database=YourDatabase;User Id=beacon_reader;Password=…;Encrypt=True;TrustServerCertificate=False`. Beacon refuses a connection that does not ([Transport security](https://moberghr.github.io/beacon/features/data-sources/#transport-security-tls)). ## Check it works - [ ] The app starts. No `Implementation of IBeaconScheduler is required.`, no encryption-key error. - [ ] In SQL Server, the `beacon` schema holds Beacon's tables and `beacon.__EFMigrationsHistory` lists the migrations. - [ ] `/beacon/setup`, the first admin and `curl -k https://localhost:/beacon/api/health` behave as in the other recipe. - [ ] The Hangfire tables exist (schema `HangFire`) and the Hangfire server is running. - [ ] After you create a subscription, `subscription-` is in the Hangfire recurring jobs. After you delete it, the entry is gone. - [ ] Create a subscription with a short schedule and wait for its run: the Hangfire job succeeds, with no "not marked as system work" warning in the log. ## If something goes wrong | Symptom | Cause and fix | |---|---| | A warning that a scope is not marked as system work, and the job does nothing | The job did not call `MarkAsSystemWork()` before Beacon code. Check that every wrapper method does. | | `NotImplementedException` mentioning scheduling or background enqueue | You use a feature (data contracts, AI documentation, AI actors, MCP evals) whose `IBeaconScheduler` member you have not implemented. | | A failed check notifies twice | Hangfire retried it. Keep `[AutomaticRetry(Attempts = 0)]` on the wrapper class. | | The migration fails with a permission error | The login cannot create the schema or tables. Create the `beacon` schema as an administrator and call `UseBeacon` without `createSchema`. | | Startup fails on `Beacon:Schema` | The schema is fixed at `beacon`. Remove the key. | | Jobs run twice with several instances | Hangfire stores recurring jobs in its database and fires each once across servers. Check that every instance uses the same Hangfire storage. | ## Next - [Embed Beacon in an existing ASP.NET Core app](https://moberghr.github.io/beacon/cookbooks/embed-postgresql-warp/): the steps this recipe leaves out. - [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/): the scheduler contract and every job Beacon expects. - [Metadata database](https://moberghr.github.io/beacon/integrate/database/): both providers, migrations and generating SQL scripts. - [Production checklist](https://moberghr.github.io/beacon/integrate/production/): proxy, TLS, key rotation and backups. --- # Sign in to Beacon with Microsoft Entra ID > Single sign-on for Beacon's UI with Microsoft Entra ID: an app registration, a tenant allow-list, guests blocked, MFA required, Entra app roles mapped to functional permissions, and a break-glass account when local passwords are off. Source: https://moberghr.github.io/beacon/cookbooks/entra-sso/ | | | |---|---| | **You end up with** | People in your tenant sign in to Beacon's UI with **Continue with single sign-on**. Other tenants and guests are refused, MFA is required, new users arrive with no access until an Admin grants a role, and one emergency account still signs in with a password. | | **Time** | about 30 minutes | | **You need** | A running Beacon host with user management on and the first admin created ([Embed Beacon](https://moberghr.github.io/beacon/cookbooks/embed-postgresql-warp/) or the sample host), permission to register an application in your Entra tenant, and a way to set host configuration and secrets. | | **Applies to** | Beacon 5.0 · Full Beacon | :::tip[Let your AI do it] Part of this recipe is clicking in the Entra admin center, so the agent edits your host's configuration and guides you through the portal. Paste this into Claude Code or another coding agent, in the repository of your Beacon host. ```text wrap Read https://moberghr.github.io/beacon/cookbooks/entra-sso.md and follow it step by step in this repository. Ask me before every decision it marks as one. Edit only non-secret configuration; tell me which user-secrets or environment variables to set and let me type the client secret myself: never ask me to paste it into this chat. Do not change anything in my Entra tenant: tell me what to click and wait for me. Show me the result of each "Check it works" item, and do not turn local password sign-in off until I have confirmed the break-glass sign-in. ``` ::: ## Before you start | Decision | Options | Default | |---|---|---| | Who may sign in | Anyone in your tenant, or only members of a group or holders of an app role (`RequiredGroups`, `RequiredRoles`) | Your tenant, plus a group if you can | | Role a new user gets | None, or a role such as `Viewer` (never `Admin` or a functional role: startup fails) | None: an Admin grants access after the first sign-in | | MFA | `RequireMfa` on or off | On, after you have seen `mfa` in a test sign-in | | Local passwords | On, or off with one break-glass account | Off once SSO works | | Entra app roles to Beacon permissions | Map them, or assign functional roles on the Users page | Assign on the Users page unless your identity team owns the roles | ## 1. Register the application in Entra In the Microsoft Entra admin center, open **App registrations** and choose **New registration**. 1. **Name**: `Beacon`. **Supported account types**: single tenant. 2. **Redirect URI**: platform **Web**, value `https://beacon.example.com/signin-oidc`. This is your app's public origin plus Beacon's `CallbackPath`, which defaults to `/signin-oidc`. It is **not** under `/beacon`. Register one per environment (add `https://localhost:/signin-oidc` for development). 3. Under **Certificates & secrets**, create a client secret. Copy its **value** now. 4. From **Overview**, note the **Application (client) ID** and the **Directory (tenant) ID**. 5. Under **Token configuration**, add the optional claim `acct` to the ID token. Beacon uses it to tell a guest from a member; without it a guest is recognised only when the token names a foreign identity provider, and Beacon logs a warning once. Beacon uses the authorization code flow with PKCE and does not keep the tokens. ## 2. Configure Beacon Store the secret outside source control: ```bash dotnet user-secrets set "Beacon:Authentication:Oidc:ClientSecret" "" # elsewhere: the environment variable Beacon__Authentication__Oidc__ClientSecret ``` Then the rest, in `appsettings.json` (the keys are the ones in `src/Beacon.SampleProject/appsettings.json`): ```json { "Beacon": { "Authentication": { "Oidc": { "Enabled": true, "Authority": "https://login.microsoftonline.com//v2.0", "ClientId": "", "AllowedTenants": [ "" ], "BlockGuests": true, "RequireMfa": false } } } } ``` Your host must call `builder.Services.AddBeaconOidcAuthentication(builder.Configuration);`. The [embed recipe](https://moberghr.github.io/beacon/cookbooks/embed-postgresql-warp/#4-register-beacon-and-map-its-endpoints) and the sample host already do. What these keys do (the full table is in [Sign-in & identity](https://moberghr.github.io/beacon/integrate/authentication/#oidc--sso)): | Key | Effect | |---|---| | `Enabled`, `Authority`, `ClientId`, `ClientSecret` | All four are required once enabled, or the host does not start. | | `AllowedTenants` | Entra tenant ids (`tid`) that may sign in. Required: with SSO on and no tenant listed the host does not start. `AllowAnyTenant: true` is only for providers that issue no `tid`, and logs a warning with an Entra authority. | | `BlockGuests` | Default `true`. Refuses Entra B2B guests. | | `RequiredGroups`, `RequiredRoles` | When either is set, the user needs at least one listed group object id or app role. Entra puts groups in the token only if you add a groups claim under **Token configuration**. | | `DefaultRoleName` | The role a user gets at their first sign-in. Unset: no role. Cannot be `Admin` or the role of a functional permission. | | `RequireMfa`, `AcceptedAcrValues` | See step 4. | | `CallbackPath` | Change it only if you register a different redirect URI. | Admission runs before Beacon looks anyone up or creates a user: tenant first, then the guest block, then the group or role requirement. A person who fails any of them never gets a Beacon user. Behind a reverse proxy, configure `UseForwardedHeaders` so Beacon builds an `https` redirect URI ([Production checklist](https://moberghr.github.io/beacon/integrate/production/#reverse-proxy)). ## 3. Sign in and grant the first roles 1. Restart the host. The login page now shows **Continue with single sign-on**. 2. Click it and sign in with an account from your tenant. The first sign-in creates the user; with no `DefaultRoleName` the page says **Your account has no access yet.** 3. Sign in as the admin you created at first-run setup, open **Users**, edit the new user and assign a role. Beacon matches a returning user by the token's `sub` and issuer, and it always takes roles from its own store: it removes every role, group and `roles` claim the token carries before building the session. **An Entra role or group named `Admin` does not make anyone a Beacon Admin.** While Beacon manages users, Admin standing comes from the stored user's live roles, and only an existing Admin can assign the Admin role. A UserAdmin cannot. Nobody is provisioned before the first-run setup has created the super admin. ## 4. Require MFA `RequireMfa: true` refuses an SSO sign-in unless the ID token's `amr` claim contains `mfa`, or its `acr` claim is one of `AcceptedAcrValues`. Enforce MFA in Entra with a Conditional Access policy, and let Beacon check that it happened. 1. Sign in once with `RequireMfa` off and look at the ID token's `amr` claim. [Single sign-on admission](https://moberghr.github.io/beacon/security/user-management/#single-sign-on-admission) advises requesting `amr` as an optional ID-token claim under **Token configuration**. 2. When `mfa` is there, set `"RequireMfa": true`. A sign-in without it lands on the login page with a message that the sign-in did not use multi-factor authentication. `RequireMfa` covers the interactive SSO sign-in only. It does not apply to bearer tokens, MCP callers, API keys or password sign-in, break-glass included. Require MFA for tokens in Entra. ## 5. Map Entra app roles to functional permissions (optional) Beacon's six functional permissions (`Approver`, `Auditor`, `SecurityAdmin`, `UserAdmin`, `DataSourceAdmin`, `MigrationOperator`) can come from Entra app roles at sign-in. They carry no project data and are never `Admin`, `Editor` or `Viewer`. 1. In the app registration, open **App roles** and create roles, for example with the value `Beacon.Auditor`. Assign people or groups to them under **Enterprise applications**, **Users and groups**. 2. Map them: ```json { "Beacon": { "Authorization": { "IdpRoles": { "Enabled": true, "Map": { "Beacon.Auditor": "Auditor", "Beacon.SecurityAdmin": "SecurityAdmin" }, "ClaimLifetimeMinutes": 60 } } } } ``` A mapped permission is added to the session when the user signs in and is never stored. It counts for `ClaimLifetimeMinutes` (default 60, at most 1440), does not show on the Users page and never reaches an API key; signing in again renews it. The host does not start when `Enabled` is true with an empty `Map`, or a value that is not one of the six permission names. More in [Access control](https://moberghr.github.io/beacon/integrate/access-control/). ## 6. Turn local passwords off and keep a break-glass account Do this last. Confirm first that SSO works and that the super admin from setup can sign in with a password. ```json { "Beacon": { "UserManagement": { "LocalLogin": { "Enabled": false, "BreakGlassUserNames": [ "" ] } } } } ``` With `Enabled: false` the login page shows no password form, and only the listed accounts may still sign in with a password. A name is a stored, enabled, internal user name, matched exactly and case-sensitively; at most three. Each break-glass sign-in logs `BreakGlassSignIn` (event 7119) as a warning: treat it as an incident. The host does not start if local login is off and nobody could sign in. Open `/beacon/login?local=1` to reach the password form. `HostOwnsSignIn` is not for this recipe. It is for a host whose own identity signs people in and which decides whom Beacon provisions: it removes first-run setup and the super admin. Leave it off ([User management](https://moberghr.github.io/beacon/security/user-management/#when-the-host-owns-sign-in)). ## Check it works - [ ] The host starts. No `Beacon:Authentication:Oidc is Enabled but ...` error. - [ ] `curl -s https://beacon.example.com/beacon/api/auth/sso` answers `{"enabled":true,...}`, and the login page shows **Continue with single sign-on**. - [ ] An account from your tenant signs in and the user appears on **Users** with no role (or `DefaultRoleName`). - [ ] An account from another tenant, and a guest, are sent back to `/beacon/login` with "Your account is not permitted to use Beacon." - [ ] With `RequireMfa` on, a sign-in without MFA is refused with the multi-factor message, and one with MFA works. - [ ] With `IdpRoles` on, a user holding `Beacon.Auditor` can open the pages that permission opens. - [ ] With local login off, the break-glass account signs in at `/beacon/login?local=1`, and every other password sign-in is refused. ## If something goes wrong | Symptom | Cause and fix | |---|---| | Entra shows `AADSTS50011` (reply URL does not match) | The registered redirect URI is not exactly `/signin-oidc`, including `https` and the port. Behind a proxy, fix forwarded headers so Beacon sends `https`. | | Startup: `... admits every tenant` | List your tenant in `AllowedTenants`. | | Startup: `DefaultRoleName cannot be the Admin role ...` | Pick `Viewer` or `Editor`, or leave it unset. | | Startup: local login is off and nobody could sign in | List a break-glass user name, or turn SSO on, or set `LocalLogin:Enabled` back to `true`. | | Everyone is sent back with "Your account is not permitted ..." | Tenant, guest, group or role admission failed. The host log has one Warning with the reason, the tenant and a hash of the subject, never the name. | | "Single sign-on failed" | The token was not valid for the app (wrong client secret, expired secret, wrong authority). Check the host log for `OIDC remote failure`. | | Signed in but "Your account has no access yet." | The new user has no role. An Admin assigns one on **Users**, or set `DefaultRoleName`. | | First SSO sign-in is refused before setup | Nobody is provisioned until first-run setup has created the super admin. Complete setup first. | | An Entra `Admin` role does nothing in Beacon | By design: roles come from Beacon's store. Assign the Admin role on **Users**. | | Mapped permission disappeared | It lasts `ClaimLifetimeMinutes` after the sign-in. Sign in again. | ## Next - [Sign-in & identity](https://moberghr.github.io/beacon/integrate/authentication/): every OIDC option, sessions, bearer tokens and API keys. - [Authorization](https://moberghr.github.io/beacon/security/authorization/): global roles, functional permissions and project access. - [User management](https://moberghr.github.io/beacon/security/user-management/): roles, first-run setup and the Users page. - [Connect Claude through Entra](https://moberghr.github.io/beacon/mcp/entra-callers/): MCP callers signing in with Entra tokens, which `Oidc:McpJwksEndpoint` and `Beacon:Mcp:Callers` handle separately from this sign-in. --- # Alert Slack or Teams when a nightly check fails > A saved query that returns the rows breaking a rule, a nightly subscription that posts them to Slack or Microsoft Teams, and a task that opens on failure and closes itself when the check passes again. Source: https://moberghr.github.io/beacon/cookbooks/alert-on-failed-check/ | | | |---|---| | **You end up with** | A nightly check on one of your databases. When it finds rows that break a rule, a message lands in a Slack or Teams channel and a task opens; when a later run finds none, the task resolves itself. | | **Time** | about 20 minutes | | **You need** | A running Beacon with a [data source](https://moberghr.github.io/beacon/features/data-sources/) for the database (or the rights to add one), a Slack or Teams channel you may add a webhook to, and the right to create queries and subscriptions. An Admin or SecurityAdmin creates the recipient. | | **Applies to** | Beacon 5.0 · Full Beacon | :::tip[Let your AI do it] This recipe is done in Beacon's web UI, so the agent guides you instead of doing it. Paste this into Claude or another assistant. ```text wrap Guide me through https://moberghr.github.io/beacon/cookbooks/alert-on-failed-check.md one step at a time. Read it first. Ask me for the database, the rule to check and the channel before each step that needs them. After each step, ask me to confirm what I see on screen against the recipe's "Check it works" list before moving on. Never ask me to paste a webhook URL or a connection string into this chat: I type those into Beacon myself. ``` ::: ## Before you start | Decision | Options | Default | |---|---|---| | What counts as a failure | A query that returns **the rows that break the rule**; any row is a failure | Required: Beacon has no separate "check condition" | | When to message you | **When result count changes**, **Every execution** or **When result count increases** | **When result count changes**: one message when the problem appears or changes, not one every night | | Channel | Slack, Microsoft Teams, or both as two recipients | Required | | Track it as a task | On or off | On: the task is the record of whether anyone dealt with it | A subscription has no pass or fail setting of its own. It runs a query and looks at the row count. So write the check as a query that returns the offenders, and an empty result means the check passed. ## 1. Add the data source Skip this if the database is already connected. 1. Open **Data Sources** and click **New data source**. 2. Pick the **Data source type**, enter a **Name** and the connection details, and pick the **Projects** to link it into. 3. Click **Test connection**, then **Create data source**. Use a read-only login: Beacon only ever reads. Details and TLS requirements are in [Data Sources](https://moberghr.github.io/beacon/features/data-sources/#create-a-data-source). ## 2. Write the check as a query 1. Open **Queries**, click **New query**, and enter a **Query name** and a **Description**. The name is what appears in the alert title. 2. In the first step, set a **Step name**, the **Target database** (your data source) and the **SQL query**. 3. Click **Save & run** to see the result. ```sql -- Orders marked completed but not paid: any row is a failure. SELECT id, status, payment_status FROM orders WHERE status = 'completed' AND payment_status <> 'paid' ``` - [ ] A healthy database returns no rows. - [ ] The columns are ones you are happy to post to a channel. Beacon masks columns it detects as PII before rows leave, but leave out what the channel does not need. Keep the window in the SQL itself (`created_at >= now() - interval '1 day'`). A task resolves on a run with zero rows, so a window that is empty only because the night was quiet closes it too early. See [Queries](https://moberghr.github.io/beacon/features/queries/) for parameters and multi-step checks. ## 3. Create the Slack or Teams recipient A recipient is one destination on one channel. Only an Admin or a SecurityAdmin creates one. 1. Create the webhook: - **Slack:** create an app with **Incoming Webhooks** on, add a webhook to the channel, copy its URL. - **Teams:** in the channel, create a Workflows (Power Automate) flow that starts when a webhook request is received and posts the card it receives. Copy its URL. Old `outlook.office.com` connector URLs are refused. 2. In Beacon, open **Recipients** and click **Add recipient**. 3. Set **Notification type** to **Slack** or **Teams**, enter a **Name**, and paste the URL into **Slack webhook URL** or **Teams webhook URL**. Click **Create recipient**. Beacon stores the URL encrypted and shows it masked afterwards. It must be HTTPS on a vendor host (`hooks.slack.com`, or Teams hosts on `*.logic.azure.com`, `*.environment.api.powerplatform.com` or `*.webhook.office.com`). The full rules are in [Destination policy](https://moberghr.github.io/beacon/features/notifications/#destination-policy). :::note[Four-eyes approval of destinations] If your Beacon has the [approval workflow](https://moberghr.github.io/beacon/security/approvals/) on, a new recipient receives nothing until a second person (an Approver or Admin other than you) approves exactly where it delivers, on **Approvals** under **Recipients**. The dialog then says "Submitted for approval". Ask the approver to compare the fingerprint on the **Recipients** page with yours. Until it is approved, a run that has no other recipient is recorded as **Skipped**. See [Approval of new and changed destinations](https://moberghr.github.io/beacon/features/notifications/#approval-of-new-and-changed-destinations). ::: - [ ] The recipient appears in the **Recipients** list and is not marked **Awaiting approval**. ## 4. Schedule the check 1. Open the query and click **Add subscription** (or open **Subscriptions** and click **New subscription**). 2. **Query:** set the **Cron expression**. `0 2 * * *` runs every night at 02:00. Fill in the query's parameters, if any. 3. **Notify:** set **Send notification** (see the table above), pick your recipient under **Recipients**, and tick **Create a task and keep it open while the query returns rows**. 4. **Results:** leave **Show query text in notification** off unless the channel should see the SQL. 5. Click **Create subscription**. You cannot edit a subscription's schedule or trigger afterwards. To change them, create a new subscription and archive the old one. Recipients can be added and removed later. The subscription runs as you. If you lose access to the query's data source, its runs are skipped; an Admin can **Change owner**. See [Subscriptions](https://moberghr.github.io/beacon/features/subscriptions/#scheduled-runs-run-as-the-owner). ## 5. Test it On the subscription's page, click **Test now**. It runs the subscription once, as you, and delivers to its recipients without changing the schedule. To see a failure, run a test while the check returns rows (insert a bad row in a test database, or temporarily loosen the SQL). To see it pass, run a test on a clean result. ## What the message contains - **Title:** `[Beacon] {data source} - {query name}`. - **Rows:** a preview of the result's first rows, after PII masking, and the total row count. Slack shows up to 25 rows and 5 columns as a text table; Teams shows an Adaptive Card with the first 10 rows and 3 columns. - **The query text**, only if you turned **Show query text in notification** on. - **A link back to Beacon**, when the host has set a base URL that ends in `/beacon`. Rows that reach recipients are masked by the PII settings of the projects that link the data source. See [What a run reads and delivers](https://moberghr.github.io/beacon/features/subscriptions/#what-a-run-reads-and-delivers). ## 6. Watch it - **Tasks** lists one open task for the subscription while the check keeps failing. Assign it to yourself, then **Resolve** it with notes when you have fixed the cause. A run with zero rows resolves it automatically with "Auto-resolved: Query returned 0 results"; a later failure opens a new task. See [Tasks](https://moberghr.github.io/beacon/features/tasks/). - **Control Tower** shows the subscription's health (Green, Amber, Red, **Stalled**), its success rate and its open tasks. A run with no rows counts as a success. See [Control Tower](https://moberghr.github.io/beacon/features/control-tower/). - **Notifications** lists every run with its status. Only **Sent** reaches the channel; **No results**, **Below threshold** and **Silenced** notify nobody by design. Over the REST API, the same steps are `POST /beacon/api/recipients`, `POST /beacon/api/queries`, `PUT /beacon/api/queries/{id}`, `POST /beacon/api/subscriptions` and `POST /beacon/api/subscriptions/{id}/execute`. See [Routes](https://moberghr.github.io/beacon/reference/routes/). ## Check it works - [ ] **Test now** on a failing check posts a message to the channel with the title `[Beacon] {data source} - {query name}`. - [ ] A task for the subscription is listed under **Tasks** as **Unresolved**. - [ ] **Test now** on a clean result posts nothing, the run shows **No results**, and the task is **Resolved**. - [ ] The subscription is **Green** in **Control Tower** after its first scheduled run. ## If something goes wrong | Symptom | Cause and fix | |---|---| | Saving the recipient is refused | The URL is outside the destination policy: not HTTPS, not a vendor host, or a retired Teams connector URL. The message names the problem. | | **Test now** answers `409` | Every recipient waits for approval. Approve it on **Approvals**, or attach one that is approved. | | Run shows **Failed** | Delivery to a recipient failed. Beacon does not retry; check the webhook, then use **Test now** or wait for the next run. | | Run shows **Silenced** | The trigger did not fire: with **When result count changes**, the count equals the last run's. Pick **Every execution** if you want a message each night. | | Run shows **Skipped** | The owner may no longer run the query, the query change is not approved, or every recipient waits for approval. The run's reason says which. | | The link in the message is broken | The base URL is missing `/beacon`. An Admin sets it under **Admin Settings**. | | The task closed but the problem persists | The query returned zero rows on one run. Widen its time window. | ## Next - [Subscriptions](https://moberghr.github.io/beacon/features/subscriptions/): triggers, thresholds, attachments and anomaly detection - [Notifications](https://moberghr.github.io/beacon/features/notifications/): every channel, webhook bodies and delivery history - [Approvals](https://moberghr.github.io/beacon/security/approvals/): four-eyes review of queries and destinations - [Score a table with a data contract](https://moberghr.github.io/beacon/cookbooks/data-contract/): scheduled quality rules with a score and alerts --- # Score a table with a data contract > A data contract on one table: rules for freshness, volume, nulls, uniqueness and more, evaluated on a schedule into a 0 to 100 score, with an alert to Slack, Teams or email when the score drops. Source: https://moberghr.github.io/beacon/cookbooks/data-contract/ | | | |---|---| | **You end up with** | A contract on one table with a handful of rules, evaluated on a cron schedule. Each evaluation stores a weighted score, the table shows a trend, and a drop below your threshold alerts a channel. | | **Time** | about 20 minutes | | **You need** | A PostgreSQL, SQL Server or MySQL [data source](https://moberghr.github.io/beacon/features/data-sources/) whose metadata is loaded, and the right to run saved SQL on it through a project. For alerts, a [recipient](https://moberghr.github.io/beacon/features/notifications/#recipients). A Custom SQL rule needs an Admin. | | **Applies to** | Beacon 5.0 · Full Beacon | :::tip[Let your AI do it] This recipe is done in Beacon's web UI, so the agent guides you instead of doing it. Paste this into Claude or another assistant. ```text wrap Guide me through https://moberghr.github.io/beacon/cookbooks/data-contract.md one step at a time. Read it first. Ask me which table to cover and which problems matter most (stale data, missing rows, nulls, duplicates, orphans) before proposing rules, and show me each rule's JSON configuration for approval before I type it in. After each step, ask me to confirm what I see against the "Check it works" list. Do not suggest a Custom SQL rule unless I am an Admin and a built-in rule type cannot express the check. ``` ::: ## Before you start | Decision | Options | Default | |---|---|---| | Table | One table per contract: a schema and a table of the data source | Required | | Rules | Freshness, Volume, Null rate, Uniqueness, Referential, Range, Pattern, Custom SQL | Start with two or three | | Schedule | A 5-field cron expression | `0 */6 * * *` (every 6 hours) | | Alert | **Alert on failure** and a **Failure threshold (%)** | On, 80 | Data contracts evaluate on PostgreSQL, SQL Server and MySQL only. On another engine the rules fail. The table and each rule's columns must exist in the data source's stored metadata: refresh the metadata first when the table is new. ## 1. Open the Data Quality page Open **Data Quality** and click **New contract**. ## 2. Describe the contract | Field | What to enter | |---|---| | **Contract name** | For example `Orders freshness and volume` | | **Data source** | Your database data source | | **Schema name** / **Table name** | For example `public` / `orders` | | **Cron expression** | `0 */6 * * *`, or `0 3 * * *` for nightly | | **Failure threshold (%)** | `80` is the default; alerts fire when the score is below it | | **Enabled** | On, so the schedule is active | | **Alert on failure** | On, then pick **Notification recipients** | ## 3. Add rules Click **Add rule** for each. Each has a **Name**, **Type**, optional **Column**, **Severity**, **Weight** and a **Configuration (JSON)**. You do not write the schema or table: Beacon takes them from the contract. | Type | Checks | Configuration | |---|---|---| | **Freshness** | Newest timestamp is not older than a limit | `{"column": "updated_at", "maxAgeMinutes": 60}` | | **Volume** | Row count in range | `{"minRows": 100}` | | **Null rate** | NULL percentage at most a limit | `{"column": "email", "maxNullPercent": 5}` | | **Uniqueness** | No duplicate values | `{"column": "id"}` | | **Referential** | Every value exists in a reference table | `{"column": "user_id", "referenceTable": "users", "referenceColumn": "id"}` | | **Range** | Numbers between `min` and `max` | `{"column": "age", "min": "0", "max": "150"}` | | **Pattern** | Every value matches a pattern | `{"column": "email", "pattern": "^[^@]+@[^@]+$"}` | | **Custom SQL** | Any check you can write as SQL | See [Custom SQL rules](#custom-sql-rules) | Pattern behaves differently by engine: a regular expression on PostgreSQL and MySQL, `LIKE` wildcards on SQL Server. **Severity** multiplies a rule's influence (Low ×1, Medium ×2, High ×3, Critical ×4) and **Weight** (0.1 to 10, default 1) tunes it within a severity. Make the rules whose failure you cannot tolerate Critical. A passing Critical freshness rule and a failing Low volume rule, both weight 1, score `(100×4 + 0×1) / (4+1) = 80%`. Most rules score 100 or 0; Null rate and Range lose points in proportion to the overrun. See [Scoring](https://moberghr.github.io/beacon/features/data-quality/#scoring). Click **Create**. An enabled contract is scheduled at once. ## 4. Evaluate it now On the contract's page, click **Evaluate now**. It runs every enabled rule once as you, stores the evaluation and updates the table's score. It sends no alert: only scheduled runs alert. The page's tabs show the result: - **Rules**: the rule set. - **Evaluations**: the history, newest first, with the overall score and pass and fail counts. - **Latest results**: per rule, pass or fail, score, expected and actual value and message. A rule that cannot run (a bad configuration, a missing column, a timeout of 60 seconds) is recorded as failed with score 0, and the other rules still run. ## 5. Alert on a drop With **Alert on failure** on and a recipient attached, a **scheduled** evaluation scoring below the threshold sends a notification titled `[Data Quality] {contract name}` to each recipient. It carries the overall score, the threshold, the table, pass and fail counts and every failed rule with its expected and actual value. To check it end to end, set **Failure threshold (%)** to 100 with a rule that currently fails, wait for the next scheduled run (or set a near cron time), and watch the channel. Then put the threshold back. The table's trend is **Improving**, **Stable** or **Degrading** by more than 1 point against the previous score. See [Table scores and trends](https://moberghr.github.io/beacon/features/data-quality/#table-scores-and-trends). ## Custom SQL rules Use one when no built-in type fits. Only an **Admin** can create, change or delete a contract that contains one. 1. Add a rule of type **Custom SQL**. 2. Put a single read-only `SELECT` in the configuration: ```json {"sql": "SELECT CASE WHEN COUNT(*) = 0 THEN 1 ELSE 0 END AS passed, COUNT(*) AS actual_value, 'orders without a customer' AS message FROM orders WHERE customer_id IS NULL"} ``` - It must return `passed` (`1` or `0`). `score` (0 to 100), `actual_value` and `message` are optional. - Do not set your own `LIMIT`, `TOP` or `FETCH`: Beacon reads one row. - Name only the data source's own database. Other qualifiers are refused. - PII in `actual_value` and `message` is masked before it is stored or alerted. :::note[Approval] If your Beacon has the [approval workflow](https://moberghr.github.io/beacon/security/approvals/) on, a Custom SQL rule that is new or whose SQL changes waits for a second person (an Approver or Admin other than you) and is not evaluated until then. The save says "Submitted for approval", the rule shows **Awaiting approval**, and the reviewer decides on **Approvals** under **Custom SQL rules**. Evaluations that leave a waiting rule out are marked **Partial** and do not replace the table's score. See [Approval of Custom SQL](https://moberghr.github.io/beacon/features/data-quality/#approval-of-custom-sql). ::: ## Over the REST API The same contract can be managed under `/beacon/api/data-quality`: `POST /contracts` to create, `POST /contracts/{id}/evaluate` to run now, `GET /contracts/{id}/evaluations` for the history and `GET /overview` for scores per data source. A scoped API key needs the Execute scope for writes. See [Data Quality → REST API](https://moberghr.github.io/beacon/features/data-quality/#rest-api). ## Check it works - [ ] The contract page shows **Latest score** after **Evaluate now**, and **Evaluations** lists the run. - [ ] **Latest results** shows each enabled rule as pass or fail with an actual value. - [ ] The **Data Quality** overview shows the table with its score. - [ ] After a scheduled run below the threshold, the recipient's channel gets `[Data Quality] {contract name}`. ## If something goes wrong | Symptom | Cause and fix | |---|---| | Saving says "Table … is not in the data source's metadata" | The table or a column is not in the stored metadata. Refresh the data source's metadata, or fix the spelling. | | "Data contract's data source must be a database type" | The data source is not a database (for example an API). Pick a database data source. | | A rule fails with "Rule type … is not supported on …" | The engine is not PostgreSQL, SQL Server or MySQL. | | "Invalid rule configuration" | The JSON is incomplete for the type. Compare it with the table above. | | "Custom SQL must not set its own LIMIT, TOP or FETCH" | Remove the outer row limit. | | "Rule timed out." | The check ran over 60 seconds. Narrow it or index the column. | | No alert despite a low score | Alerts need **Alert on failure**, a recipient, a score strictly below the threshold and a scheduled run. A recipient waiting for approval receives nothing. | | **Owner not authorized** on the contract | The owner may no longer run saved SQL on the data source, so evaluations are skipped. An Admin or UserAdmin uses **Change owner**. | | "Every enabled rule … is a Custom SQL rule waiting for approval" | Approve it on **Approvals**, or enable a rule that does not wait. | ## Next - [Data Quality](https://moberghr.github.io/beacon/features/data-quality/): ownership, scoring and every rule option - [Alert Slack or Teams when a nightly check fails](https://moberghr.github.io/beacon/cookbooks/alert-on-failed-check/): the query-based alternative - [Notifications](https://moberghr.github.io/beacon/features/notifications/): channels and destination approval - [Approvals](https://moberghr.github.io/beacon/security/approvals/#migration-jobs-custom-sql-rules-and-recipients): reviewing Custom SQL rules --- # Ship the MCP audit and security events to your SIEM > Send Beacon's MCP audit events and security log events to your log pipeline, turn on OpenTelemetry traces and metrics, set retention and fail-closed audit, and read the MCP audit in the UI. Source: https://moberghr.github.io/beacon/cookbooks/audit-to-siem/ | | | |---|---| | **You end up with** | The `Beacon.Audit` log stream and the security log events in your SIEM, traces and metrics in your collector, retention and content locks set, and alerts on the signals that matter | | **Time** | about 30 minutes | | **You need** | A running Beacon host ([Integrate Beacon](https://moberghr.github.io/beacon/integrate/) or the [MCP gateway](https://moberghr.github.io/beacon/mcp/gateway/)), a log pipeline that reads container stdout (CloudWatch, a collector or a SIEM agent), and for traces an OpenTelemetry collector | | **Applies to** | Beacon 5.0 · Both. The MCP audit (events 9100 to 9103, the UI page, tracing and metrics) applies to the gateway too. The security audit trail (9104 to 9114) and its page are full Beacon: the gateway serves no security audit API or page | :::tip[Let your AI do it] Paste this into Claude Code (or another coding agent) in your Beacon host's repository. ```text wrap Read https://moberghr.github.io/beacon/cookbooks/audit-to-siem.md and follow it step by step in this repository. Ask me which log pipeline and collector we use, and what retention policy applies. Never commit keys or endpoints with credentials, never turn off an audit setting, and show me the result of each "Check it works" item. ``` ::: ## Before you start | Decision | Options | Default | |---|---|---| | Fail closed | Withhold a tool result when its audit row was not written (`Audit:Required`), or return it anyway | Fail closed where an unaudited answer is not acceptable | | MCP audit retention | Keep forever, or purge rows older than N days | Forever (`null`): set it to your policy | | Audit content | Keep the question and SQL of every call (`RequireContent`), let each project decide, or keep none (`ForceNoContentRetention`) | Each project decides. The two locks cannot both be on | | Traces and metrics | OTLP to a collector, or none | None: Beacon registers no exporter | ## 1. Turn on structured logs Beacon writes its audit events as structured log events under the category **`Beacon.Audit`**. JSON console logs carry the event id, category, level and trace ids with no OpenTelemetry needed: ```json { "Logging": { "Console": { "FormatterName": "json", "FormatterOptions": { "IncludeScopes": true } } } } ``` Make sure the log level for the category lets Information through (`"Beacon.Audit": "Information"` under `Logging:LogLevel`): the copies of committed events are Information, and a filtered category drops them. Ship the stream with whatever reads your container's stdout, such as the `awslogs` driver into CloudWatch Logs. ## 2. Know what to route `Beacon.Audit` holds identifiers, counts and digests only: never parameters, SQL, a question or a raw error message. | EventId | Name | Level | Meaning | |---|---|---|---| | 9100 | `McpToolAudited` | Information | One per MCP tool call, with `Tool`, `ProjectId`, `CallerKind`, `CallerHash`, `DurationMs`, `Rows`, `ErrorClass`, `TraceId`; `Persisted` says whether the row was written | | 9101 | `McpToolResultWithheld` | Warning | A result was withheld because its audit row failed (needs `Audit:Required`) | | 9102 | `McpAuditPurged` | Information | The MCP audit retention purge ran | | 9103 | `McpAuditRead` | Information | Someone read the MCP audit | | 9104 | `SecurityAuditRecorded` | Information | A security audit event was committed (copy of the row; match by `EventGuid`) | | 9105 | `SecurityAuditRead` | Information | The security trail was read or verified | | 9106 | `SecurityAuditPurged` | Information or Warning | The security trail's retention purge ran | | 9107 | `SecurityAuditWriteFailed` | Error | A security event could not be stored | | 9108 | `SecurityAuditVerificationFailed` | Error | Stored security events failed verification | | 9109 | `SecurityAuditCheckpoint` | Information | One per scheduled verification run: keep these, they anchor the trail outside the database | | 9110 | `SecurityAuditPurgeFailed` | Error | The security trail's purge did not run | | 9111 | `McpAuditVerificationFailed` | Error | Stored MCP audit rows failed verification | | 9112 | `McpAuditCheckpoint` | Information | One per MCP audit verification run: keep these with the 9109 checkpoints | | 9113 | `AuditTableVerificationFailed` | Error | Stored execution log or settings history rows failed verification (`Store` names the table) | | 9114 | `AuditTableCheckpoint` | Information | One per sealed table per verification run: keep these with the 9109 checkpoints | The full field list is in [MCP observability and audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/#the-beaconaudit-log-event-catalogue) and [Shipping to a SIEM](https://moberghr.github.io/beacon/security/audit-trail/#shipping-to-a-siem). The structured security log events 7101 to 7123 (membership and role changes, sessions, break-glass sign-ins, refused access, row data sent to an LLM, and more) have their own ids: route them too. The list is in the [security model](https://moberghr.github.io/beacon/security/#12-audit-and-security-events). Alert at least on 7119 (break-glass), 7113 and 7114 (role changes, new users), 7115 (write targets) and 7116 (a statement tried to end SQL Server's read-only transaction). The copy is best-effort: the database is the source of truth, and the 9107 lines and the 9109 checkpoints let you notice when the two disagree. ## 3. Set retention, content and fail-closed audit ```jsonc { "Beacon": { "Mcp": { "Audit": { "Required": true, // withhold a result whose audit row was not written "RetentionDays": 400, // purge MCP audit rows older than this; null keeps them forever "RequestIdHeader": "X-Request-Id", "RequireContent": false // true pins question and SQL retention on, for everyone } }, "Audit": { "RetentionDays": 400 // security audit trail; at least 365; unset keeps events forever } } } ``` - `Required: true` replaces the result with *Result withheld: the audit record for this call could not be written.* and logs 9101. The query has already run by then: the guarantee is that no unaudited data leaves Beacon. - `RequireContent: true` keeps the question and the SQL of every statement a call ran until the retention purge. Review that against your privacy policy first. It contradicts `Beacon:Mcp:ForceNoContentRetention`, and the host does not start with both on. The gateway pins no content retention unless you set `RetainToolContent = true`. - Both purges are jobs you schedule: the MCP purge is `IMcpAuditRetentionService.PurgeExpiredAsync`, the security trail has `ISecurityAuditRetentionService.PurgeExpiredAsync` and an hourly `ISecurityAuditVerificationService.VerifyNewEventsAsync`. Set `Beacon:Audit:IntegrityKey` from a secret store before you change the encryption key. See [Retention](https://moberghr.github.io/beacon/security/audit-trail/#retention) and [Keys and rotation](https://moberghr.github.io/beacon/security/audit-trail/#keys-and-rotation). ## 4. Turn on traces and metrics (optional) Beacon registers no exporter. Reference the OpenTelemetry packages your platform needs and add Beacon's sources: ```csharp builder.Services.AddOpenTelemetry() .ConfigureResource(x => x.AddService(serviceName: "your-app")) .WithTracing(x => x .AddAspNetCoreInstrumentation() .AddBeaconInstrumentation() // source: "Beacon" .AddBeaconMcpSdkTracing() // source: "Experimental.ModelContextProtocol", status descriptions cleared .AddOtlpExporter()) .WithMetrics(x => x .AddAspNetCoreInstrumentation() .AddBeaconInstrumentation() // meters: "Beacon" and "Experimental.ModelContextProtocol" .AddOtlpExporter()); ``` Set `OTEL_EXPORTER_OTLP_ENDPOINT` (and `OTEL_SERVICE_NAME`) in the environment: Beacon reads no endpoint or credential itself. Beacon tags the MCP SDK's own `tools/call` span with `beacon.project.id`, `beacon.caller.hash`, `beacon.error.class` and similar, so one MCP call is one trace. Each audit row carries its `TraceId`, which joins the SIEM, the trace and the row. Alert on any increment of these counters: | Counter | Meaning | |---|---| | `beacon.mcp.audit.write_failures` | An MCP audit row was not written | | `beacon.security_audit.write_failures` | A security event was not stored, or a purge did not run | | `beacon.security_audit.verification_failures` | Stored security events failed verification | `beacon.mcp.tool.calls` (tagged `gen_ai.tool.name`, `beacon.outcome`) and `beacon.mcp.tool.rows` give usage and error rates. Tool names on the metric are bucketed (`q_*`, `api_*`) so caller-chosen names cannot create unbounded series. `beacon.tool.input` (the tool input on the span) stays off unless both `Beacon:Telemetry:CaptureContent` and the project's content retention allow it. A collector that receives OTLP and exports to X-Ray and CloudWatch metrics: ```yaml receivers: otlp: protocols: http: grpc: processors: batch: exporters: awsxray: awsemf: service: pipelines: traces: receivers: [otlp] processors: [batch] exporters: [awsxray] metrics: receivers: [otlp] processors: [batch] exporters: [awsemf] ``` The collector holds the cloud credentials, never Beacon. ## 5. Read the audit in Beacon The **MCP Audit** page (`/beacon/mcp-audit`, in the sidebar for people with the Auditor permission) lists tool calls with time, tool, project, caller, user, duration, rows and outcome. Filter by From and To (UTC), Tool, Project id, User id and Caller hash. The gateway's admin UI serves it too. Reading the audit is recorded: 9103 and, in full Beacon, a security audit event. The same data is available as a paged JSON export for an Auditor or an Admin (sign in as one): ```text wrap GET https://beacon.example.com/beacon/api/mcp/audit?from=2026-10-01T00:00:00Z&to=2026-10-21T00:00:00Z&pageSize=100 ``` `from` and `to` are required and at most 93 days apart; `pageSize` is 1 to 500. Rows carry `contentHash` and their `statements`, with `sql` only where the project retained content. Full Beacon also has a **Security Audit** page for the security trail. ## Check it works - [ ] Make one MCP tool call. Your SIEM shows a `Beacon.Audit` event 9100 for it, and its `TraceId` matches the **MCP Audit** row. - [ ] In full Beacon, change a project member. A 9104 event arrives with an `EventGuid` that matches the Security Audit row. - [ ] With a collector configured, a tool call shows one trace tagged with `beacon.project.id`. - [ ] With `Audit:Required` on, a tool result is never returned without its row; a failure shows as 9101 and an increment of `beacon.mcp.audit.write_failures`. - [ ] The retention jobs run: 9102 (and 9106 in full Beacon) appear after the purge. ## If something goes wrong | Symptom | Cause and fix | |---|---| | No `Beacon.Audit` events in the SIEM | The category is filtered below Information, or the log driver does not ship stdout. Check `Logging:LogLevel` for `Beacon.Audit`. | | 9100 events but no 9104 | 9104 is full Beacon only and best-effort: it is missing when the category is filtered or the process stops between commit and log. | | Host fails to start with an options error | `RetentionDays` must be greater than 0 (security trail: 365 to 36500), `RequestIdHeader` must be a valid header token, and `RequireContent` cannot be on with `ForceNoContentRetention`. | | No `CorrelationId` on old rows | Rows written before it existed carry none; every later row and 9100 event has one Beacon minted. | | `UpstreamRequestId` is empty | The header is absent, longer than 128 characters, or has characters outside `[A-Za-z0-9._:-]`. | | No traces | `AddBeaconInstrumentation()` is not on the tracer provider, or `OTEL_EXPORTER_OTLP_ENDPOINT` is unset. | | Logs in the dispatched endpoint carry no trace id | Host endpoint dispatch does not flow `Activity.Current`. See [Trace continuity](https://moberghr.github.io/beacon/mcp/observability-and-audit/#trace-continuity-into-host-endpoint-dispatch). | | The audit page is missing from the sidebar | The user lacks the Auditor permission. | ## Next - [MCP observability and audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/): every field, tag and instrument. - [Security audit trail](https://moberghr.github.io/beacon/security/audit-trail/): the event catalogue, integrity and verification. - [Production checklist](https://moberghr.github.io/beacon/integrate/production/#audit-and-observability): what to set before real users. --- # MCP > Beacon's Model Context Protocol server: one governed endpoint through which Claude, Claude Code, Cursor, VS Code and other MCP clients read your catalog, run read-only SQL, call approved saved queries and your own endpoints, every call project-scoped, masked and audited. Source: https://moberghr.github.io/beacon/mcp/ Beacon serves a **Model Context Protocol (MCP)** server at `/beacon/mcp`. Through it, AI assistants (Claude, Claude Desktop, Claude Code, Cursor, VS Code and any other MCP client) work with your data and your application under the same rules as people: they reach only the projects they are given, run read-only SQL only, see masked values only, and every call they make is audited. :::tip[Hand this section to your AI] The whole MCP section is published as one file for agents. Paste this into Claude Code, Cursor or any coding agent: ```text wrap Read https://moberghr.github.io/beacon/llms-mcp.txt. It is the documentation of Beacon's MCP server. Then help me with: . Ask me before every decision the docs mark as one, and never put an API key or secret in a committed file. ``` ::: ## Two ways to use it | | **Beacon's MCP server** over your databases | **MCP gateway** for your own application | |---|---|---| | Agents reach | Your data: the catalog, documentation, read-only SQL, approved saved queries, and optionally your endpoints | Your application: a set of its endpoints, run as the signed-in person, and its documents | | Who connects | API keys with the Execute scope, limited to projects, or mapped Microsoft Entra callers | Your application's own users, signed in with Microsoft Entra (delegated tokens) | | SQL | Yes, read-only, through the execution gate | None: no data sources, no SQL tools | | Package | `Moberg.Beacon.MCP` on a full Beacon | `Moberg.Beacon.Gateway`, about thirty lines of `Program.cs` | | Start with | [Connect a coding agent](https://moberghr.github.io/beacon/cookbooks/connect-coding-agent/) | [MCP gateway for your application](https://moberghr.github.io/beacon/mcp/gateway/) | Both run the same server, filters and audit. They combine: a full Beacon can expose host endpoints as tools, and a gateway host can add Beacon's data product later. ## The tools Eight built-in tools, plus the ones you add. Every tool is published with a title and safety annotations, so clients can show what a call does before it runs. | Tool | What it does | Typical use | |---|---|---| | `get_context` | Project overview: data sources, schemas, tables, documentation status | First call of a session | | `search` | Keyword search over tables, columns and documentation, fused with semantic matching when local embeddings are on | "Where do we keep refunds?" | | `get_documentation` | Documentation at project, data-source or table level | Reading what a table means | | `get_query_context` | The grounding `ask` uses: schema with join paths, glossary and verified examples | Before an agent writes its own SQL | | `dry_run` | Validates SQL through every gate without running it | Checking SQL before `query` | | `query` | Runs read-only SQL against one data source, row-capped and masked | Getting the numbers | | `ask` | Answers a natural-language question: routing, grounded SQL generation, gated execution | One-shot business questions | | `feedback` | Rates an answer; a `correct` verdict becomes a verified example only after an Admin reviews it | Teaching Beacon what good looks like | | `q_` | An approved [saved query](https://moberghr.github.io/beacon/mcp/saved-query-tools/) published as a tool, with typed parameters | Repeatable, reviewed reports | | `api_` | One of your own [HTTP endpoints](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/), run as the caller through its own authorization | Actions and reads in your application | The gateway serves your `api_` tools plus `get_context`, `search` and `get_documentation` over your documents. Every tool, argument and limit is described on [Server & tools](https://moberghr.github.io/beacon/mcp/server/). ## How a call is governed ```mermaid flowchart LR C["MCP client
Claude · Cursor · VS Code"] -->|"Bearer API key
or Entra token"| A["Caller
Execute scope
no browser sessions"] A --> P["Project access
evaluated on every call"] P --> T["Tool"] T --> G["SQL gate
read-only · AST validator
cross-database · row cap"] G --> X["Execution
READ ONLY transaction
statement timeout"] X --> M["PII masking"] M --> R["Result"] T -.-> AU[("MCP audit
every call, refusals included")] ``` - **Credentials, not sessions.** `/beacon/mcp` accepts an API key with the Execute scope or a mapped bearer token. A signed-in browser session gets `403`; people try the tools in the **MCP playground** inside Beacon instead. - **Projects are the boundary.** The caller's projects are decided on every call, so a revoked membership takes effect on the next call. A call that names a project outside them is refused, and the refusal is audited. - **Read-only, enforced twice.** Every statement passes the [SQL execution gate](https://moberghr.github.io/beacon/security/#6-the-sql-execution-gate), and PostgreSQL and MySQL also run it in a read-only transaction. Read-only enforcement and PII detection are deployment locks, on by default, that even an Admin cannot switch off at runtime. - **Audited, including refusals.** Every tool call is recorded with its caller, timing and outcome. What is kept (questions, SQL, error text) is a per-project setting that the host can pin. The full model is on [Security](https://moberghr.github.io/beacon/security/). ## Connect a client With an API key (`sk-sem_…`, Execute scope, created under **API Keys** in Beacon): ```bash claude mcp add --transport http beacon https://beacon.example.com/beacon/mcp \ --header "Authorization: Bearer $BEACON_MCP_API_KEY" ``` | Client | How | |---|---| | Claude Code, Cursor, VS Code | An API key in a header: [Connect a coding agent](https://moberghr.github.io/beacon/cookbooks/connect-coding-agent/) | | claude.ai, Claude Desktop, Cowork | A custom connector that signs in with Microsoft Entra: [Claude connector with Entra](https://moberghr.github.io/beacon/cookbooks/claude-connector-entra/) | | Agent platforms and services | Entra tokens mapped to user or system callers: [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/) | | Anything else | Streamable HTTP and JSON-RPC by hand: [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/#check-it-works) | Then ask it something. Example prompts for the first session are in the [prompt library](https://moberghr.github.io/beacon/cookbooks/prompts/#ask-your-data). ## Make it yours | You want agents to | Use | |---|---| | Run a reviewed report with parameters, not free SQL | [Saved query tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/) | | Call your application's endpoints as the signed-in person | [Host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/), or the whole [gateway](https://moberghr.github.io/beacon/mcp/gateway/) | | Understand your EF Core model | [Host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/) | | Read your own Markdown documentation | [Host docs](https://moberghr.github.io/beacon/mcp/host-docs/) | | Write better SQL from your glossary and verified examples | [Grounding & knowledge base](https://moberghr.github.io/beacon/mcp/knowledge-base/) | | Use fewer built-in tools, or other server instructions | `AddBeaconMcp(x => …)`: see [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/#wiring) | ## Operate it - **See what agents did:** the MCP audit, the `Beacon.Audit` log events, OpenTelemetry spans and metrics, and retention: [Observability & audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/), or the recipe [Ship the MCP audit to your SIEM](https://moberghr.github.io/beacon/cookbooks/audit-to-siem/). - **Pin what nobody may change:** read-only, PII detection, content retention, row data to the LLM and numeric ceilings under `Beacon:Mcp`: [Deployment locks and ceilings](https://moberghr.github.io/beacon/mcp/server/#deployment-locks-and-ceilings). - **Check a deployment from outside:** `beacon-doctor mcp https://your-app/beacon/mcp` walks the sign-in path the way a client does: [Verify a deployment](https://moberghr.github.io/beacon/mcp/gateway/#8-verify-a-deployment). - **Test your tools in CI:** `Moberg.Beacon.Testing` gives a real MCP client over your test server: [Expose an endpoint as a tool](https://moberghr.github.io/beacon/cookbooks/endpoint-as-tool/). ## Packages | Package | For | |---|---| | `Moberg.Beacon.MCP` | The MCP server on a full Beacon (`AddBeaconMcp`, `MapBeaconMcp`, `MapMcpDiscovery`) | | `Moberg.Beacon.Gateway` | The MCP gateway for your own application (`AddBeaconMcpGateway`, `UseBeaconMcpGateway`, `MapBeaconMcpGateway`) | | `Moberg.Beacon.Testing` | Test tokens, test users and an MCP client for a gateway host's tests | | `Moberg.Beacon.Doctor` | The `beacon-doctor` .NET tool that checks a deployed endpoint | ## In this section - [Server & tools](https://moberghr.github.io/beacon/mcp/server/): every tool, argument, guardrail, limit and setting - [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/): wiring, discovery documents, API keys or Entra, client set-up, checking by hand - [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/): user and system callers from Microsoft Entra, and Claude's custom connector - [Observability & audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/): audit rows, log events, spans, metrics, retention - [MCP gateway for your app](https://moberghr.github.io/beacon/mcp/gateway/): the whole gateway recipe - [Saved query tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/), [Host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/), [Host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/), [Host docs](https://moberghr.github.io/beacon/mcp/host-docs/), [Grounding & knowledge base](https://moberghr.github.io/beacon/mcp/knowledge-base/) --- # MCP server and tools > A Model Context Protocol server that lets AI assistants query your data sources, search your catalog, and read documentation over authenticated HTTP. Source: https://moberghr.github.io/beacon/mcp/server/ Beacon exposes a **Model Context Protocol (MCP)** server at `/beacon/mcp`. AI clients and agents use it to search your catalog, read documentation, and run read-only SQL against the data sources of a project, under the same project access, guardrails and audit as the rest of Beacon. ## Overview The MCP server is **project-centric**: each API key is scoped to one or more projects, and all tools automatically resolve which data sources, schemas, and documentation to use based on the active project. Project resolution is **stateless**: it is computed per call, with no session memory. If the key is restricted to a single project, that project is used automatically; if it can reach several, pass `project_id` on every call (the error lists the candidates when you don't). **Which projects a caller reaches** is decided on every call by the same [project access](https://moberghr.github.io/beacon/security/authorization/#project-access) rules as the REST API and the playground. The projects the caller may read: - an **API key** reaches the projects of its list its owner may still use; a key without a list (created before lists were required) reaches every project its owner may use; - a mapped **Entra system caller** reaches its configured projects, restricted or not; - a mapped **Entra user caller** reaches its configured projects within its Beacon user's access. A restricted project only while the user is a member; without a Beacon user, its configured open projects only; - a **browser session** (cookie or SSO, Beacon's or the host's) is refused on `/beacon/mcp` with `403` (browsers use the [MCP Playground](#mcp-playground)), and a call without an HTTP request behind it (a host-wired stdio transport) reaches no project. **What a caller may do in a project** follows its project role there, as in the playground: the catalog tools (`get_context`, `search`, `get_documentation`, `get_query_context`) need Read, while the SQL-carrying tools (`ask`, `query`, `dry_run`) and `feedback` work ad hoc in the project and need Query. An Editor or above in it. A Viewer (a Viewer member of a restricted project, or a caller whose global role is Viewer) reads the catalog but runs no SQL there: such a call answers "Access denied: you may read project N but not run ad-hoc queries in it." and is audited, and an `ask` or `query` refused this way records its signal like any other failed call. **No MCP caller curates grounding.** Curating (reviewing learned patterns, applying documentation patches, verifying relationships, editing or regenerating documentation) needs [Curate](https://moberghr.github.io/beacon/security/authorization/#project-roles), which Beacon grants only to a signed-in user's session (a project admin member of the project, or an Admin): never to an MCP caller, user or system, whatever its scope, and never to an API key. Golden examples are an Admin's alone. Over MCP, `feedback` records a verdict and curates nothing by itself (see [Who curates grounding](https://moberghr.github.io/beacon/mcp/knowledge-base/#who-curates-grounding)). A membership revoked, a project restricted or a user disabled takes effect on the caller's next call, in the same session. Saved-query tools are visible only in the caller's projects where it may run saved SQL (`Run`), and host endpoint tools only when the caller may read the host's project. A tool call that names a project the caller does not reach answers "Access denied: you do not have access to project N."; one that names none when the caller reaches no project answers "No projects are available to you." Both are audited. **What you can do through MCP:** - Ask natural language questions and get SQL + results back - Pull the exact grounding context Beacon uses for SQL generation (schemas with sample values when a project turns them on, verified join paths, and human-verified examples) and write your own SQL (`get_query_context` → `dry_run` → `query`) - Validate SQL through every safety gate without executing it - Execute direct SQL queries against any data source in your project - Search tables, columns, and documentation by keyword, fused with embedding-based semantic matching when a local embedder is enabled - Retrieve AI-generated documentation for your project, data sources, or individual tables - Record feedback on answers. Verdicts are recorded for an Admin to review, and an Admin may promote a `correct` answer to a verified example that grounds future SQL generation ## Quick start ### 1. Create an API key Go to **API Keys** (`/beacon/api-keys`) and choose **Generate key**: - Choose the **scope** `Execute`. The MCP endpoint requires it, because its tools can run SQL; `Read` keys reach only the read-only REST API. Creating an `Execute` key needs the Editor role or above. - Pick the **projects** the key may use; at least one is required. The key reaches them only while you, its owner, still have access to them (see [Project restrictions](https://moberghr.github.io/beacon/security/api-keys/#project-restrictions)). - Copy the key (`sk-sem_…`). It is shown once; Beacon stores only its SHA-256 hash. ### 2. Connect a client Every client sends the key as a bearer token: `Authorization: Bearer sk-sem_…`. For Claude Code, add Beacon to the project's `.mcp.json`: ```json { "mcpServers": { "beacon": { "type": "http", "url": "https:///beacon/mcp", "headers": { "Authorization": "Bearer ${BEACON_MCP_API_KEY}" } } } } ``` or from the command line: ```bash claude mcp add --transport http beacon https:///beacon/mcp --header "Authorization: Bearer sk-sem_…" ``` Other clients, Entra sign-in and manual JSON-RPC calls are covered in [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/). ### 3. Try it > "What tables are available in my project?" > > "How many orders were placed last week?" > > "Show me the schema for the customers table" When your key reaches more than one project, the client must pass `project_id` on every call. ## Connection details | Property | Value | |----------|-------| | **Endpoint** | `/beacon/mcp` | | **Transport** | Streamable HTTP + JSON-RPC 2.0 (via `ModelContextProtocol.AspNetCore`) | | **Authentication** | Required: `Authorization: Bearer sk-sem_…` (an API key with the `Execute` scope) or a mapped [Entra ID](https://moberghr.github.io/beacon/mcp/entra-callers/) bearer token | | **Sessions** | Hybrid: stateless-protocol clients need no session; initialize-handshake clients get an `Mcp-Session-Id` (idle timeout 30 minutes) | Map the server with `app.MapBeaconMcp()`, which mounts `/beacon/mcp` behind the Execute-scope policy (registered by `AddBeaconApiAuthorization()`): an API key or mapped Entra caller without the `Execute` scope gets `403`. Do not map it with `app.MapMcp("/beacon/mcp").RequireAuthorization()`, because the default policy admits any authenticated caller. The MCP layer also refuses a caller that lacks the `Execute` scope and a caller that is not authenticated, over HTTP or over a transport without an HTTP request (such as stdio). A host that maps the route with a weaker policy therefore still fails closed, but it loses the `403` at the door. A tool call answers a tool error (no tool runs, and the refusal is audited like any tool call), and every other request except `initialize` and `ping` (`tools/list`, resources, prompts, completions) answers a JSON-RPC error. Clients exchange JSON-RPC messages with the single `/beacon/mcp` endpoint over the Streamable HTTP transport; the server streams responses back on the same connection. **Sessions and load balancers.** Clients that speak the stateless protocol are served without a session, so any instance behind a plain load balancer can answer any request. Clients that start with the `initialize` handshake get an `Mcp-Session-Id`, and their follow-up requests must reach the instance that holds that session: run several instances behind session affinity for them. A session idle for 30 minutes is closed, and the client starts a new one. **Protocol niceties the server publishes:** - The `initialize` response carries **server instructions** describing the recommended tool workflow, and every tool is published with a human-readable **title** and **annotations** (`readOnlyHint`, `idempotentHint`, `destructiveHint`, `openWorldHint`) so clients can reason about safety before calling. - `query` and `ask` responses include machine-readable **`structuredContent`** (columns, rows, row count, truncation flag, plus the generated SQL and `signal_id` for `ask`) alongside the markdown text, so agents do not have to parse tables back out of prose. - **Truncated results say so explicitly**: row-capped query results, paged search results, and concise documentation exports all end with a note stating that more data exists and how to get it (raise `max_rows`, repeat with the next `offset`, or pass `response_format: "detailed"`). ## Connecting from remote clients Remote MCP clients that connect by URL need one URL: ```text https:///beacon/mcp ``` Authentication is a bearer token in the `Authorization` header: a Beacon API key with the `Execute` scope (`Read` keys cannot reach the MCP tools), for clients that let you set a header. Clients that sign in with OAuth find the discovery documents below; Beacon does not run an OAuth server, so those clients sign users in with Microsoft Entra ID (see [Connect Claude](https://moberghr.github.io/beacon/mcp/entra-callers/#connect-claude-claudeai--claude-desktop-custom-connector)). Agent platforms can instead call with **Microsoft Entra ID tokens**, either on behalf of a user or as a system identity. Beacon configuration decides their projects and scope. See [Entra ID callers on MCP](https://moberghr.github.io/beacon/mcp/entra-callers/). ### Discovery endpoints Beacon publishes anonymous, read-only discovery metadata so remote clients can bootstrap a connection without documentation: | Endpoint | What it serves | |----------|----------------| | `/.well-known/oauth-protected-resource` | [RFC 9728](https://datatracker.ietf.org/doc/rfc9728/) protected-resource metadata: the resource identifier (the canonical `/beacon/mcp` URL), `scopes_supported`, bearer-header auth. By default `scopes_supported` is `["Execute"]` and, when SSO is enabled (`Beacon:Authentication:Oidc`), `authorization_servers` lists the configured OIDC authority; API-key-only deployments omit it. A host whose callers sign in with Microsoft Entra ID publishes the Entra issuer and scopes instead through `Beacon:Mcp:ProtectedResource`. See [Connect Claude](https://moberghr.github.io/beacon/mcp/entra-callers/#connect-claude-claudeai--claude-desktop-custom-connector) | | `/.well-known/oauth-protected-resource/beacon/mcp` | The same document at the RFC 9728 path-inserted variant clients derive from the `/beacon/mcp` resource path | | `/.well-known/mcp/server-card.json` | A server card (per the draft SEP-2127 proposal): server name and version, transport (`streamable-http`), endpoint URL, auth summary, and the eight built-in tools with their titles (saved-query and host endpoint tools are not listed) | Unauthenticated requests to `/beacon/mcp` answer `401` with a `WWW-Authenticate: Bearer resource_metadata="…"` header pointing at the metadata document, so OAuth-capable clients discover the auth requirements from the challenge itself (RFC 9728 §5.1). A bearer value that is not a valid token (malformed, wrong algorithm, unknown signing key, expired) gets the same `401`, with `error="invalid_token"` added to the challenge. When the JWKS endpoint cannot be reached and no keys are cached, a bearer JWT gets `503` with `Retry-After` instead: the token was never judged, and a `401` would make an OAuth client discard it and sign in again unnecessarily. Behind a reverse proxy, set `Beacon:PublicBaseUrl` (e.g. `https://beacon.example.com`) so the discovery documents advertise the public origin; when unset, the URLs are derived from the incoming request and the responses are not cacheable. A registry manifest for the official [MCP registry](https://registry.modelcontextprotocol.io) ships in `deploy/registry/server.json` with a publishing runbook alongside it. Publishing is a deliberate out-of-band step (DNS verification of the namespace domain). ## Tools The MCP server exposes **8 built-in tools**. Every tool is published with `destructiveHint: false` and `openWorldHint: false`; every `project_id` parameter is optional (see [Overview](#overview)). | Tool | Title | `readOnlyHint` / `idempotentHint` | Needs in the project | Recorded as | |---|---|---|---|---| | `get_context` | Project Overview | true / true | Read | audit | | `search` | Search Catalog | true / true | Read | audit | | `get_documentation` | Get Documentation | true / true | Read | audit | | `get_query_context` | Get Grounding Context for a Question | true / true | Read | audit | | `ask` | Ask a Data Question | true / **false** | Query | audit + signal | | `query` | Run Read-Only Query | true / true | Query | audit + signal | | `dry_run` | Validate SQL Without Executing | true / true | Query | audit + signal | | `feedback` | Record Answer Feedback | **false** / true | Query | audit (writes the verdict on an existing signal) | Further tools appear next to them: | Tools | What they are | Above the limit | |---|---|---| | `q_` | Approved saved queries a SecurityAdmin exposed; see [Saved Query Tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/) | `search_saved_queries` + `run_saved_query` above 25 (`SavedQueryTools.NamedToolLimit`) | | `api_` | Host endpoints the host marked as tools (read-only, or write tools behind an opt-in); see [Host Endpoint Tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/) | `search_api` + `call_api` above 20 (`NamedToolLimit`) | ![MCP Playground](https://moberghr.github.io/beacon/img/screenshots/mcp-playground-dark.png) ### `get_context` Get an overview of the project: data sources, schemas, tables, quality scores, and documentation status, plus the [host application tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/) the caller's tool list offers (none for a credential that cannot call them, no write tool for a system caller) and the project's [imported documents](https://moberghr.github.io/beacon/mcp/host-docs/), up to 50 of each listed. **This is the recommended starting point** for understanding what the project offers. A project without data sources, such as an [MCP gateway's](https://moberghr.github.io/beacon/mcp/gateway/), leaves out the data source, documentation and repository counts and lists its tools and documents. | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `project_id` | integer | No | Specify project if your API key has access to multiple projects | | `format` | string | No | `overview` (default) or `agents_md`: a deterministic project brief for an agent workspace's `AGENTS.md` (see [Host Docs](https://moberghr.github.io/beacon/mcp/host-docs/#on-the-mcp-surface)) | **Example response** (markdown): ```markdown # Project: E-Commerce Analytics **Data Sources:** 2 **Documentation:** Generated **Repositories:** 1 ## Data Sources ### production-db (ID: 4) - **Type:** PostgreSQL - **Tables:** 45 - **Quality:** 87% - **Code References:** 124 - **Schemas:** public (40 tables), audit (5 tables) ### analytics-api (ID: 7) - **Type:** Api - **Endpoints:** 12 - **Code References:** 9 - **Tags:** users (7 endpoints), orders (5 endpoints) ``` ### `ask` Ask a natural language question about your data. Beacon auto-detects the right data source(s), generates SQL, executes it, and returns results. | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | `question` | string | **Yes** | None | Natural language question (e.g., "How many orders were placed last week?") | | `project_id` | integer | No | None | Specify project if needed | | `execute` | boolean | No | `true` | Set to `false` to get the generated SQL without executing it | **How it works:** 1. **Routing phase**: The LLM determines which data source(s) to query (skipped for single-source projects). Each source the model names counts once; one `ask` joins at most four, and a question that spans more is refused with an error naming how many it spans (audited and signalled like every failed call), never answered from some of them 2. **SQL generation**: Generates SQL using your actual schema as context 3. **Execution**: Runs the query with safety guardrails (read-only, row limits, PII detection) **Cross-source queries:** If your question spans multiple data sources, Beacon queries each source separately and joins results in an in-memory SQLite database. **Budgets:** one `ask` call runs for at most `Beacon:Mcp:Ceilings:AskTimeoutSeconds` (default 120 seconds), its statements included, and makes at most `Ceilings:AskMaxLlmCalls` LLM calls (default 12: intent, routing, generation, self-consistency candidates, repairs and the cross-source join together). The budget counts logical calls; a provider's transport retries of one call are not counted. Self-consistency asks for no more candidates than the calls left after keeping two for repairs, and a repair with no call left is skipped, so the answer is the attempt as it stands. A call that runs out of time, or out of calls before it generated any SQL (a cross-source `ask` needs one call per source plus the join), returns an error naming the budget; it is audited and signalled like every failed call. ### `query` Execute a direct SQL query against a specific data source. Use this when you already know the exact query you want to run. | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | `datasource_name` | string | No* | None | Name of the data source | | `datasource_id` | integer | No* | None | ID of the data source (alternative to name) | | `sql` | string | No | None | SQL query (SELECT only) for database sources | | `api_query` | string | No | None | JSON query definition for REST API sources | | `max_rows` | integer | No | `100` | Maximum rows to return, capped at the project's max row limit (default 1000). A value of 0 or below is raised to 1. | | `project_id` | integer | No | None | Specify project if needed | *Either `datasource_name` or `datasource_id` is required. `query` checks every column against the project's catalog and refuses one it does not know. A data source with no extracted metadata has no catalog to check against, so `query` refuses it with *This data source has no extracted metadata (it was never scanned, metadata loading is off, or the scan found no tables); query it through MCP once it has metadata.* (audited and signalled like other refusals): turn metadata loading on and refresh its metadata first. Host data sources keep their own allow-list; `dry_run` reports a missing catalog as an advisory and still validates the rest. CloudWatch sources cannot be queried through `query` (they run Logs Insights, not SQL); Beacon's own CloudWatch features use the configured log groups only, so restrict the IAM role to them. **For REST API data sources**, pass a JSON query definition: ```json { "method": "GET", "path": "/api/users", "parameters": { "limit": 10 }, "resultMapping": { ... } } ``` ### `get_query_context` Get the grounding context Beacon assembles internally for the `ask` tool, scoped to your question, so **you** (or your agent) can write well-grounded SQL instead of delegating generation to Beacon's LLM. The context contains M-Schema table renderings *with real sample values*, join paths (verified foreign keys and inferred relationships kept apart), coverage notes when the table neighbourhood was capped, human-verified golden query examples, learned patterns from usage, and matching business-glossary terms. | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | `question` | string | **Yes** | None | The question you plan to answer with SQL. The context is retrieved and ranked against it | | `datasource_name` | string | No* | None | Name of the data source to ground against | | `datasource_id` | integer | No* | None | ID of the data source (alternative to name) | | `project_id` | integer | No | None | Specify project if needed | | `max_chars` | integer | No | `12000` | Maximum characters of context returned (min 1000, max 30000) | *If the project has exactly one data source it is auto-selected; with several, pass `datasource_name` or `datasource_id` (the error lists the candidates as `id: name` pairs). `get_query_context` needs Read in the project. Its value matches come from live probes of the data source only for a caller who may also query the project (Editor and above); a Viewer gets the matches the stored sample values already hold, and no probe runs for it. The context shows sample values and value matches only while sample-value collection and value grounding are on for the project and every project sharing the data source (both off by default), and never under the deployment lock `Beacon:Mcp:ForbidRowDataToLlm`; they pass the same name and value screens as everywhere else (see [Sample values](https://moberghr.github.io/beacon/features/data-sources/#sample-values)). Unlike the REST metadata endpoint, which returns stored sample values to an Admin only, `get_query_context` returns them to any caller with Read in the project, since they are what grounds its SQL. A response that carries them is recorded in the row-data egress log (event `7123`, "returned to the MCP client"; see [What reaches the LLM provider](#what-reaches-the-llm-provider)). The response opens with a header naming the data source and SQL dialect, followed by the grounding context. Sections marked **(authoritative)** are human-verified. When the context exceeds `max_chars` it is cut at the last complete section and ends with an explicit truncation note. `structuredContent` carries `data_source_id`, `data_source`, `dialect`, and `truncated`. **Intended workflow:** `get_query_context` → write your own SQL → `dry_run` to validate → `query` to execute. Use this instead of `ask` when your agent wants full control over the SQL it runs. ### `dry_run` Validate a SQL query through all of Beacon's safety gates **without executing it**. Use before `query` to catch problems for free. | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `datasource_name` | string | No* | Name of the data source to validate against | | `datasource_id` | integer | No* | ID of the data source (alternative to name) | | `sql` | string | **Yes** | The SQL query to validate (SELECT only) | | `project_id` | integer | No | Specify project if needed | *Either `datasource_name` or `datasource_id` is required. **The gates**, in order: the `gate` value of each issue: 1. **`read_only`**: the shared execution gate: the regex guardrail, the dialect-aware AST validator (DML/DDL, stacked queries, comment-hidden writes), the cross-database and system catalog rules and the row limit, plus PII detection (it reports the columns that would be masked). The issue's `code` names the layer that refused the SQL, for example `guardrail`, `ast`, `cross_database` or `system_catalog`. 2. **`schema`**: the schema-catalog column check, which catches hallucinated tables and columns without a database round-trip. A data source without extracted metadata gets an advisory issue and an invalid verdict. 3. **`provider_dry_run`**: the database's own validation (`EXPLAIN` / `sp_describe_first_result_set`); it runs only when the earlier gates found nothing to fix (a missing schema catalog alone does not stop it). An engine with no dry-run strategy reports the gate as skipped with an advisory issue, and the verdict is invalid: a validation that never ran is never counted as a pass. Issues are collected rather than first-failure-wins, so one call reports everything to fix. The response gives a per-gate verdict and, when valid, the exact SQL that would execute with the row limit applied. `structuredContent` carries `{ valid, issues: [{ gate, code, error }], executable_sql, pii_columns }`. An invalid verdict is still a successful tool call; `isError` is reserved for resolution failures. ### `get_documentation` Retrieve AI-generated documentation at three levels of detail. | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `project_id` | integer | No | Specify project if needed | | `datasource_name` | string | No | Get docs for a specific data source | | `table_name` | string | No | Get detailed docs for a specific table or API endpoint | | `schema_name` | string | No | Schema name or API tag (optional qualifier for table_name) | | `response_format` | string | No | (any response is also cut at the project's `MaxResultBytes` with a truncation note) `concise` (summary sections) or `detailed` (everything). Project level defaults to `concise`; data-source, table and document level default to `detailed` | | `document` | string | No | Path (or exact title) of a [host-imported document](https://moberghr.github.io/beacon/mcp/host-docs/); returns it in full | **Three levels:** 1. **Project level** (no parameters): Full generated project documentation. Defaults to `concise`: the export is cut at ~8,000 characters on a line boundary with an explicit truncation note; pass `response_format: "detailed"` for the full document 2. **Data source level** (`datasource_name` only): Tables, schemas, code references, quality scores. Defaults to `detailed`; `concise` omits the LLM schema context section 3. **Table level** (`table_name`): Columns with types, relationships, quality rules; `detailed` (the default) adds code references and lineage, `concise` omits them Table documentation, code references and lineage come from the resolved project only: on a data source several projects share, another project's documentation and repositories never appear. The quality rules on table level are those of the table's enabled data contracts, each as its contract's latest evaluation found it: a disabled rule, a Custom SQL rule waiting for approval and a rule added since are not listed. Each result is shown as a status Beacon writes itself, never the result's stored text: the rule's outcome in words (`3 duplicate values found`, `Data is stale`, `The rule did not complete`) with, for a failed built-in rule, the number its actual value starts with. A Custom SQL rule shows only `Custom check passed` or `Custom check failed`: its `message` and `actual_value` are whatever its query returned, and never reach the model. When the host ships documents with [`ExposeDocs`](https://moberghr.github.io/beacon/mcp/host-docs/), the project-level response ends with an *Imported documents* listing (title + path) and `document: ""` returns one of them. ### `search` Search tables, columns, and documentation across all data sources in the project. The keyword arm matches the words of the query, not the query as one string: "how do I request annual leave" looks for `request`, `annual` and `leave` (function words and single characters are dropped, at most 8 words, and words keep their underscores, so `order_items` stays one), and a result holding more of the words ranks higher. Keyword matching is fused with embedding-based semantic matching (reciprocal rank fusion) when a [local embedder](https://moberghr.github.io/beacon/mcp/knowledge-base/#local-embeddings) is configured; without one, search falls back to keyword-only rather than failing. | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | `query` | string | **Yes** | None | Search keyword (e.g., "customer", "order_date", "revenue") | | `project_id` | integer | No | None | Specify project if needed | | `max_results` | integer | No | `20` | Maximum results to return (max: 50) | | `offset` | integer | No | `0` | Result offset for paging, max `200`. Use it with `max_results` to page through large result sets | Each result line includes the item type (`[TABLE]`, `[COLUMN]`, `[DOC]`, or `[IMPORTED_DOC]` with the document path for [host-imported documents](https://moberghr.github.io/beacon/mcp/host-docs/)), the data source and schema-qualified table (plus the column name for column hits), and the description, ordered by relevance. When more matches exist beyond the current page, the response ends with an explicit note giving the `offset` to request next. ### `feedback` Record whether a previous `ask` answer was correct. The verdict is recorded on the answer's signal with the user who recorded it, and it does this: - An `incorrect` verdict keeps the answer out of the common-query and join patterns Beacon learns. - A `correct` verdict puts the answer on the [golden candidates](https://moberghr.github.io/beacon/mcp/knowledge-base/#how-cases-are-created) list (the **Golden candidates** tab of the MCP Learning page), where an Admin may promote it to a verified example (golden pair) that grounds future SQL generation for the same data source; feedback never promotes it by itself. A `correct` verdict counts in [pattern mining](https://moberghr.github.io/beacon/mcp/knowledge-base/#from-signal-to-injected-lesson) only when a curator recorded it from an interactive session (a project admin member, or an Admin, signed in); a verdict through this tool, an MCP caller or an API key, is attributed to the user it acts for but never counts as a curator's, whatever that user's roles, and neither does the asker's own. The tool's description, its answer, the server instructions and the playground's feedback form all say so. | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `signal_id` | integer | **Yes** | The `signal_id` from the `ask` response you are rating | | `verdict` | string | **Yes** | `correct` or `incorrect` | | `corrected_sql` | string | No | The corrected SQL, if you fixed it | | `note` | string | No | A short note, **only when the user explicitly asked to record one**, never a summary of the conversation | Every `ask` response that ran a data query ends with a `_signal_id: N_` marker. Pass that value back here. Conceptual answers from the knowledge base do not record a query signal and carry no marker. A signal is rated only in its own project: the signal must belong to a project, that project must be the one the call resolves to (in the playground, the selected project; on the transport, one of the caller's projects), and the caller needs Query there. Anything else answers "Query signal N not found.", the same as a signal that does not exist, and is audited. Within its project, a verdict is recorded only by the caller who asked, a project admin of that project, or an Admin; anyone else gets an error saying so, and the refusal is audited too. `POST /beacon/api/eval/feedback` is Admin-only: the whole `/beacon/api/eval` group requires an Admin. `corrected_sql` is at most 100,000 characters and `note` at most 2,000; a longer one is refused and nothing is recorded. The verdict is always recorded, with the user who recorded it. Whether `corrected_sql` and `note` are *stored* depends on the project's **Allow the feedback tool to store question and SQL text** setting. See [Content retention](#content-retention) below. When that setting is off, the verdict still lands and the rest is dropped. `corrected_sql` is checked before it is stored, in the dialect of the data source the rated answer ran on: its comments are removed, and it must pass the same read-only gate and catalog check a saved query step passes (a single read-only statement, columns that exist in Beacon's metadata; a data source Beacon holds no metadata for refuses it). Nothing runs. Comment characters inside a string literal are kept when the dialect reads them as part of it; a comment that cannot be removed safely, or characters a reviewer cannot see, refuse the SQL (see [the grounding SQL check](https://moberghr.github.io/beacon/mcp/knowledge-base/#who-curates-grounding)). Corrected SQL that cannot be kept never refuses the call: the verdict is recorded and only the SQL is dropped. This covers SQL the check refuses, SQL on a signal without a single data source (a cross-source or knowledge answer: there is no dialect to check it in), and SQL the project's retention setting does not keep. The log records the signal id only. **The answer** starts with "Feedback recorded for signal N.", says what the verdict does, and, when `corrected_sql` was given, whether it was kept: "The corrected SQL was kept with the verdict, without its comments.", or "The corrected SQL was not kept:" with the reason (the project does not retain query content; the answer did not run on a single data source; it did not pass the SQL check, with the check's reason). `POST /beacon/api/eval/feedback` answers the same as `correctedSqlRecorded` (true when corrected SQL was given and kept) and `correctedSqlNotRecordedReason` (null when it was kept or none was given); the playground's feedback form shows the reason. ## Safety & guardrails The MCP server enforces several safety measures: | Feature | Description | Default | |---------|-------------|---------| | **Read-only enforcement** | Only read-only statements run | On; locked on by default (`Beacon:Mcp:ForceReadOnly`) | | **Database names** | SQL may name only its data source's configured database (catalog, project); other qualifiers are refused | Always on | | **Database boundary** | Functions and procedures that reach other servers, files or storage, SQL-text extension functions, PostgreSQL `COPY … PROGRAM` / server-file `COPY`, Databricks file paths and four-part names are refused | Always on | | **Shared data sources** | A source several live projects link runs, and is grounded, with the strictest guardrails among them | Always on | | **Row limits** | `query`: `max_rows` (default 100) capped at the max row limit; `ask` on one data source: 100 rows, or the max row limit when that is lower; cross-source: the max row limit | Max row limit 1000 | | **PII detection** | Detects and masks sensitive values in results | On; locked on by default (`Beacon:Mcp:ForcePiiDetection`) | | **Query timeout** | Each statement `query`, `ask` and the saved-query tools run may take the project's **Statement timeout** (other Beacon paths use `Beacon:Query:StatementTimeoutSeconds`, default 120) | 30 seconds | | **Plan cost** | On a PostgreSQL data source, each statement `query`, `ask` and the saved-query tools run is first planned with `EXPLAIN (FORMAT JSON)` (never `ANALYZE`; the same read-only, single-statement path and statement timeout as the statement, with its parameters) and refused before it runs when the planner's estimated total cost exceeds the project's **Max EXPLAIN cost**, with a message naming the estimate and the limit; the refusal is audited. An estimate that fails fails the call as the statement would. Other engines ignore the setting | No limit | | **Result size** | The text and the structured result of `query`, `ask` and the saved-query tools are held together to the project's **Max result size**: the structured result takes at most half, the text (heading, markdown table and notes) the rest. A larger result returns fewer rows, marked as truncated by size (`truncated` plus `rows_omitted_for_size`, `rowsOmittedForSize` on saved-query tools, and a note in the text). Not the deployment's `Beacon:Query:MaxResultBytes` (256 MiB), the budget a result is read under from the data source, past which the run fails (see [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/#query-execution-row-cap)) | 262,144 bytes | | **Concurrent queries** | One credential (an API key, else a mapped caller, else a Beacon user) may run up to the project's **Max concurrent queries per API key** `query`, `ask`, `dry_run` and saved-query calls at once on each Beacon instance; another call is refused at once with *Too many concurrent queries for this credential; retry shortly.* and audited | 4 | | **Request rate** | One API key or mapped caller may send `Beacon:RateLimits:Mcp` requests a minute to `/beacon/mcp` on each Beacon instance; more answer HTTP `429` (`application/problem+json`, `Retry-After`) before the MCP server reads them. `ask` is also limited per caller (`Beacon:RateLimits:Ask`, MCP and playground together); another question is refused with *Too many questions for this credential; retry in N seconds.* and audited. See [Request limits](https://moberghr.github.io/beacon/getting-started/configuration/#request-limits) | 60 requests, 10 questions a minute | | **Audit logging** | Every tool call is recorded by `McpAuditService` with user, timing, and parameters (see [MCP observability and audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/)) | Always on | | **Usage signals** | `ask`, `query`, and `dry_run` calls are recorded by `McpSignalService` to feed the learning loop | Always on | | **Content retention** | Whether questions, SQL, and error text are persisted at all, per project, lockable deployment-wide | Retained | **Database names.** `ask`, `query`, `dry_run` and the saved-query tools refuse SQL that names another database than the data source's own (`database.schema.table` on SQL Server, Azure Synapse, PostgreSQL and Snowflake, `database.table` on MySQL, `catalog.schema.table` on Databricks, `project.dataset.table` on BigQuery) with the gate code `cross_database`, also when read-only enforcement is off. The allowed database is the one the data source's connection configures; a source without one, or whose connection settings cannot be read, accepts no database qualifier. Unqualified and schema-qualified names always pass. The same always-on stage refuses `dblink…`, `lo_…`, `xp_…`, `OPENROWSET`, `OPENQUERY`, `OPENDATASOURCE`, `EXTERNAL_QUERY` and the other functions that reach other servers, files or storage (on Databricks also `http_request`, `read_kafka` and the other stream readers, `reflect` and `java_method`), the PostgreSQL extension functions that run SQL text (`pg_background_…`, pgRouting's `pgr_…`), PostgreSQL `COPY … TO/FROM PROGRAM` and `COPY` to or from a server file, Databricks file paths and SQL Server four-part names, so turning read-only enforcement off never opens them. See [Database names in SQL](https://moberghr.github.io/beacon/security/authorization/#database-names-in-sql). **System catalog.** The same tools refuse SQL that reads the engine's catalog, statistics or query history (`information_schema`, `pg_catalog` and `pg_…` relations, `sys`, Snowflake `QUERY_HISTORY` and `RESULT_SCAN`, BigQuery `INFORMATION_SCHEMA`, …) with the gate code `system_catalog`, also when read-only enforcement is off. `get_context` and `get_query_context` describe a source's tables and columns from Beacon's metadata. **Shared data sources.** The guardrails a tool applies are the data source's: when several live projects link it, read-only enforcement and PII detection are on if any of them turns them on, value grounding and sample values are off if any of them turns them off, all their custom PII patterns apply, and the lowest row limit and value-grounding probe budget win, whichever project the caller works in. The execution settings (statement timeout, result size, plan-cost limit) resolve to the lowest value the same way. `ask` applies them over every data source it routes to, and grounding (`ask`, `get_query_context`) applies the value-grounding, sample-value and PII settings. A metadata refresh, which no project asks for, collects sample values while collection is on in the effective settings of every live project that links the source (each project's override, else the global value); the global value alone decides only for a source no live project links. See [Guardrails of shared data sources](https://moberghr.github.io/beacon/security/authorization/#guardrails-of-shared-data-sources). | Setting | Strictest among the linking projects | |---------|--------------------------------------| | `enforceReadOnly`, `enablePiiDetection` | on when any project turns it on | | `enableValueGrounding`, `enableSampleValueCollection` | off when any project turns it off | | `customPiiPatterns` | every project's patterns | | `maxRowLimit`, `statementTimeoutSeconds`, `maxResultBytes`, `maxExplainCost`, `valueGroundingMaxProbes` | the lowest (`maxExplainCost`: any limit over none) | `McpAuditService` fires on every tool invocation, including the failure path, and it is never short-circuited. Audit rows and usage signals record the caller by its Beacon user (`Users.Id`): an API key's owner, a mapped caller's provisioned user, or the signed-in user in the playground, never the session's own identifier, so a caller can rate the signals of its own `ask` calls. A tool call refused for a missing `Execute` scope (or an unauthenticated caller) is audited too, with the tool name, the caller's identifiers and a permission error; when `Beacon:Mcp:Audit:Required` is set and that row cannot be written, the refusal answers with the withheld-result message instead. `McpSignalService` records the three SQL-carrying tools (`ask`, `query`, `dry_run`); the catalog tools (`get_context`, `search`, `get_documentation`, `get_query_context`) are audit-only by design. ## Content retention By default Beacon stores what an agent asked and what SQL ran. That is what makes the learning loop and the audit trail useful. For a project handling regulated or sensitive data you can turn that off without losing the audit trail itself. Switch **Retain question and SQL text in signals and audit** off for a project and Beacon keeps the **structure** of every interaction and stops persisting its **content**: | Kept | Dropped | |---|---| | Tool, project, data source, user, caller | The natural-language question | | Tables and columns referenced | Generated, corrected, and user-supplied SQL | | Row counts, timings, success flags, verdicts | Routing rationale and feedback notes | | An **error class** (see below) | Free-text database error messages | | The audit row itself: always | The audit row's raw parameters | | Keyed hashes of the parameters and of each statement run | The text of each statement the call ran | An audit row is still written for **every** tool call, success or failure. The lock changes what the row contains, never whether it exists. While content is retained, the audit keeps the SQL of **every statement** a call ran (an `ask`'s first execution, repairs, self-consistency candidates, value-grounding probes and dry runs) beside the question; see [The statements a call ran](https://moberghr.github.io/beacon/mcp/observability-and-audit/#the-statements-a-call-ran). A deployment that must keep this evidence pins retention on with `Beacon:Mcp:Audit:RequireContent` (see [Keeping audit content](https://moberghr.github.io/beacon/mcp/observability-and-audit/#keeping-audit-content-the-requirecontent-lock)). **Error classes.** Instead of the provider's message, Beacon stores one of nine values: `schema`, `syntax`, `permission`, `timeout`, `not_found`, `validation`, `execution`, `cancelled`, `unknown`. A tool's own argument errors (an invalid, unknown or missing argument) and a host endpoint's `400` or `422` are `validation`, whatever the argument is called. You keep the ability to chart failure modes without keeping text that may quote customer data. **Audit parameters** become a structural summary: which tool ran, how many bytes of input it received, and which tables it touched: ```json { "tool": "query", "params": [{ "name": "input", "bytes": 412 }], "tables": ["orders", "customers"] } ``` **What else changes under the lock** - The `feedback` tool records the verdict but not the corrected SQL or note. Signals of a locked project carry no question, so they are not offered as golden candidates. - Pattern mining keeps schema corrections and join patterns (structure) and skips common-query and documentation-gap mining (content). Join patterns are stored without example questions or SQL, and the LLM lesson extractor is not called, because its prompt would carry the question and SQL. - The optional eval judge is forced off, and eval results are stored without generated SQL or judge commentary. **Enforcement.** Every write path applies the rule, and an EF Core `SaveChangesInterceptor` backs them up: content is stripped on the way to the database even if a future code path forgets, and it fails closed: if the settings cannot be read, the row is treated as locked. All of this is logged with identifiers only. Retention is **not retro-active**: rows written before you turned the lock on keep their content until they age out of the retention window. A row that is *updated* after the lock is on is cleaned up by the interceptor. **Outside MCP.** Turned off globally (or pinned off with `Beacon:Mcp:ForceNoContentRetention`), the same rule covers Beacon's other content stores: the SQL and stored result snapshot of a subscription run and its notifications, the statement text of the execution log (its error text becomes an error class), the analysis, findings and conversation of an AI actor, and the statement of a migration run. A project override does not reach them: they belong to no single project. The [data retention job](https://moberghr.github.io/beacon/getting-started/configuration/#data-retention-beaconretention) clears result snapshots stored before the lock, and with `Beacon:Retention:ContentDays` the content of older rows. ## SQL accuracy stack Natural-language questions through the `ask` tool don't go through a naive prompt-to-SQL pipe. Every generated query passes a layered accuracy stack: 1. **Grounded context**: before generation, Beacon assembles an M-Schema rendering (column name, type, nullability, description), *with real sample values and value matches* when a project turns sample-value collection and value grounding on (both off by default; see [What reaches the LLM provider](#what-reaches-the-llm-provider)), so filters match actual data formats (`'shipped'` vs `'SHIPPED'`), plus curated join paths, matching business-glossary terms, human-verified query examples, and lessons mined from past usage. See the [Knowledge Base guide](https://moberghr.github.io/beacon/mcp/knowledge-base/) for what goes in and how you curate it. 2. **AST read-only validation**: generated SQL is parsed into an abstract syntax tree in the data source's own dialect (PostgreSQL, SQL Server, MySQL, BigQuery, Snowflake, Databricks); SQL for a source whose dialect is unknown is refused. DML/DDL statements, stacked queries, comment-hidden writes, row-locking clauses, functions that read files, reach other servers or sleep, linked-server (four-part) names and file path references are rejected before anything reaches your database. This is defense-in-depth beyond the regex guardrail. 3. **Database read-only backstop**: on PostgreSQL and MySQL, MCP-executed SQL additionally runs inside a `READ ONLY` transaction (on PostgreSQL as a batch of exactly one command, which the driver and the server refuse to split into several statements; on MySQL also with a read-only session default and statement batches turned off), so a write that somehow slipped past both parsers is still refused by the database itself. This is defence in depth, not a replacement for a read-only login. BigQuery and Databricks have no read-only session: their connectors refuse, on the read-only path, anything their own dialect's parser does not read as a single `SELECT` in the configured project or catalog, and BigQuery additionally runs only what a dry run reports as a `SELECT`. SQL Server, Azure Synapse and Snowflake rely on the parser gates (SQL Server also runs the statement in a transaction that is always rolled back, and fails the run when the statement ends that transaction itself). On all five, the read-only login you connect with is the database-level boundary; on PostgreSQL and MySQL it is still required. The same gate and read-only path run every query Beacon executes outside MCP too: previews, subscriptions, data-quality rules, AI actors, migration reads; see [How a query runs](https://moberghr.github.io/beacon/features/queries/#how-a-query-runs). 4. **Dry-run repair loop**: if execution fails, the SQL is retried with the database error and a refreshed schema context; the error goes into the prompt redacted by its class (see [Database errors](#database-errors)); truncated or degenerate repairs are rejected rather than executed. 5. **Row limits & PII masking**: results are capped and sensitive values (emails, phone numbers, SSNs, credit cards, tokens, and custom regex patterns) are masked (`a***z`) before leaving the server. A column is masked when its own name is PII, when it is an alias over a PII column, or when it reads a JSON document or map by a literal PII key (`profile ->> 'email'`, `profile #> ARRAY['email']`, `JSON_VALUE(profile, '$.email')`, Snowflake `v:email`, `m['email']`). The masker cannot parse a Databricks `v:email` path, so a statement that runs with one has every result column masked, as has a SQL Server `FOR JSON` or `FOR XML` statement, which folds every row into one column. A key column named after a contact word (`address_id`, `zip_code_id`) holds an id and is not masked; an `EXISTS` or `IN (subquery)` test is a boolean and is not masked either. A host data source's masked columns come back as `***`, masked once and in full; when the host policy check does not pass, every result column comes back as `***`. ## Learning loop The MCP server is self-improving. `McpSignalService` records a usage signal for every SQL-carrying call, holding the question, the generated SQL, the tables and columns referenced, the routing decision, and the outcome. Recurring background jobs turn those signals into knowledge that grounds future generation: ![MCP Learning](https://moberghr.github.io/beacon/img/screenshots/mcp-learning-dark.png) | Job | Schedule | What it does | |---|---|---| | `mcp-learning-aggregate` | Every 6 hours | Mines `ask` / `query` signals into learned patterns, retires stale ones, and runs replay verification | | `mcp-learning-cleanup` | Daily 03:00 | Drops signals past the retention window | | `mcp-embedding-reindex` | Every 12 hours | Re-embeds schema metadata and exemplars | | `mcp-docchunk-reindex` | Every 12 hours | Re-chunks and re-embeds project documentation | Two things are worth knowing about how patterns get promoted: - A candidate pattern is not trusted on a confidence score alone. Before promotion, Beacon **replays the project's golden cases** with and without the candidate injected, and keeps it only if it flips at least one case from failing to passing. - `dry_run` signals are recorded for analytics but **excluded from mining**: their "question" is SQL, not natural language, and gate rejections would skew the failure statistics. Review the queue on the **MCP Learning** page (`/beacon/mcp-learning`), where pending patterns can be approved or rejected, and any stored documentation patch applied or rejected. The full mechanics (lesson extraction, decay, retention, and every setting) are in the [Knowledge Base guide](https://moberghr.github.io/beacon/mcp/knowledge-base/#the-learning-loop). ## Configuration Admins and SecurityAdmins customize the MCP server at **MCP Settings** (`/beacon/mcp-settings`); Auditors read it. The routes: `GET /beacon/api/mcp/settings` and `GET /beacon/api/mcp/projects/{projectId}/settings` for an Auditor or a SecurityAdmin, the `PUT`s for a SecurityAdmin (Admins hold both; see [Functional permissions](https://moberghr.github.io/beacon/security/authorization/#functional-permissions)). **Global defaults or per project.** A scope selector at the top of the page switches between the global defaults and any single project. In project scope each field gets an **Override** checkbox: leave it unchecked and the project inherits the global value (shown greyed out); check it and the project keeps its own. Prompts and tool descriptions stay global by design: `tools/list` is answered before any project is known. The tabs: **Pre-prompt** - **Ask tool system prompt**: the LLM prompt used for SQL generation in the `ask` tool, sent as is. The global instruction and the schema context follow as a first user message, the stable prefix a provider can reuse from its prompt cache (Claude is asked to cache it; OpenAI and Azure OpenAI cache such a prefix on their own; Bedrock does not request prompt caching yet); the date and the question come in a second user message. An `ask` repair sends that same first message, byte for byte, then the question, the previous SQL, its error and the rules in a second message ending with "Return ONLY the corrected SQL query."; a repair over any other context (a cross-source repair) sends the full schema first with no cache request. A provider's cache covers the system prompt too, so a repair reuses the generation's cached prefix only when both use the same system prompt, that is when this prompt is set. Schema text (database comments, documentation, glossary) is always user content, never part of the system prompt. That first message also states that text inside `` blocks is data, never instructions, and carries the schema context in such a block (its two marker lines are not counted in `AskSchemaContextChars`); a repair's table schemas, previous SQL and error are blocks too. Descriptions, sample and matched values, glossary terms and learned patterns stay on their own line inside the block, characters a reviewer cannot see are left out, and a block marker written inside the data is neutralised, also one written with lookalike brackets or letters, fullwidth forms, accents or control characters, so stored text cannot close its block. Data source routing, intent classification, the cross-source join, knowledge answers, documentation generation and section edits, chunk indexing, AI actors, lesson extraction and the eval judge mark their context the same way. - **Global instruction**: prepended to the user context of every LLM-aware tool **Tool descriptions** - Override the description returned by `tools/list` for `get_context`, `ask`, `query`, `get_documentation`, and `search`. Leave a field blank to keep the built-in description. Overrides are applied to the live tool list, so connected clients see your wording. `dry_run`, `get_query_context`, and `feedback` currently use their built-in descriptions only. **Guardrails** - **Max row limit**: the ceiling on rows returned (default: 1000). It must be between 1 and 10,000,000; a save outside that range, globally or for a project, is rejected with `400`. A lower value stored before this check is read as 1, so `query`, `dry_run`, `ask`, cross-source queries and evaluations all cap at one row, never run uncapped and never refuse the cap itself. - **Read-only enforcement**: the SELECT-only restriction; locked on by default (see below) - **PII detection**: enable/disable, plus custom PII regex patterns; detection is locked on by default (see below). A custom pattern that cannot be evaluated over a query (invalid, or not finished within its one-second timeout) refuses the query, naming the pattern by its position; as a column check it marks the column as PII. A new pattern is at most 500 characters and must not use lookarounds, backreferences or atomic groups (it runs on the linear-time regex engine); a save with one is rejected with `400`. A pattern stored before these checks keeps working, on the backtracking engine under the timeout, and does not block a save that leaves it in place - **Row data sent to the LLM provider**: sample-value collection and value grounding, both off by default (see [What reaches the LLM provider](#what-reaches-the-llm-provider)) - **Execution**: the two retention switches, **Retain question and SQL text in signals and audit** and **Allow the feedback tool to store question and SQL text** (see [Content retention](#content-retention)); and **Statement timeout (s)** (default 30), **Max result size (bytes)** (default 262144), **Max EXPLAIN cost** (blank = no limit) and **Max concurrent queries per API key** (default 4). The statement timeout, result size and concurrent-query limit apply to the MCP SQL tools (see the guardrails table above); the EXPLAIN cost limit is enforced on PostgreSQL data sources, where a statement whose planner-estimated total cost exceeds it is refused before it runs, and other engines ignore it - **Learning**: **Enable learning**, auto-approve threshold (stored; auto-approval is decided by replay), injection budget, and signal retention **Ranges.** Every numeric setting is checked when it is saved, globally or as a project override (an empty override inherits and is not checked); a value outside its range is rejected with `400` and a message naming the field and the range. A save names the first field out of range; fix it first, then save again to see the next. A deployment ceiling is not a bound here: a value above it is saved and clamped when the settings are read. | Setting | Allowed | |---------|---------| | `maxRowLimit` | 1–10,000,000 | | `statementTimeoutSeconds` | 1–3600 | | `learningSignalRetentionDays` | 1–36,500 | | `learningAutoApproveThreshold` | 0–1 | | `maxExplainCost` | blank (no limit), or 0–9,999,999,999,999,999.99 | | `maxResultBytes`, `maxConcurrentQueriesPerKey`, `selfConsistencyCandidateCount`, `selfConsistencyMinTables`, `exemplarTopK`, `glossaryTopK`, `docChunkTopK`, `goldenExemplarTopK`, `docChunkWindowSentences`, `learningReplayMinFlips` | at least 1 | | `learningInjectionBudgetChars`, `goldenExemplarBudgetChars`, `valueGroundingMaxProbes`, `docChunkOverlapSentences` | at least 0 (0 = none) | A statement timeout, result size or concurrency limit of 0 or less stored before these checks is read as its default (30 seconds, 262,144 bytes, 4) globally, and as no override (the global value) for a project; a statement timeout above 3600 seconds is read as 3600. Each such field is reported as clamped. The **Max result size** here bounds what an MCP tool returns; the deployment's `Beacon:Query:MaxResultBytes` bounds what is read from the data source. **Context preview** - Render the grounding context for a project exactly as the tools would assemble it. This is useful for spotting missing documentation before an agent hits it. ### Deployment locks and ceilings Some settings must be non-negotiable no matter what an administrator clicks. The `Beacon:Mcp` configuration section pins values at the host level. Read-only enforcement and PII detection are pinned **by default**: `ForceReadOnly` and `ForcePiiDetection` are `true` unless the host's configuration sets them to `false`. ```json { "Beacon": { "Mcp": { "ForceReadOnly": true, "ForcePiiDetection": true, "ForceNoContentRetention": true, "ForbidRowDataToLlm": true, "Ceilings": { "MaxRowLimit": 1000, "StatementTimeoutSeconds": 60, "MaxResultBytes": 1048576, "MaxConcurrentQueriesPerKey": 8, "MaxSqlChars": 100000, "MaxQuestionChars": 4000, "AskTimeoutSeconds": 120, "AskMaxLlmCalls": 12, "AskSchemaContextChars": 120000 } } } } ``` - A **lock** (`ForceReadOnly` and `ForcePiiDetection`, both on by default; `ForceNoContentRetention`, `ForbidRowDataToLlm` and `Audit:RequireContent`, off by default) pins the effective value for the global settings and every project. `Audit:RequireContent` pins content retention on, the opposite of `ForceNoContentRetention` (the host does not start with both on). `ForbidRowDataToLlm` pins three settings off: sample-value collection, value grounding and the eval judge (see [What reaches the LLM provider](#what-reaches-the-llm-provider)). Nobody can change a locked setting at runtime, an Admin included; only the host's configuration lifts the lock. The global page shows a locked switch disabled with a note, the project scope hides locked fields and names them in a banner, `GET /beacon/api/mcp/settings/locks` lists them (`lockedFields`, and each lock with the setting it pins in `locks`), and an API call that tries to set one gets **HTTP 409** with the RFC 7807 type `/errors/setting-locked`. - With `ForceReadOnly` on, the SQL execution gate also judges every statement read-only whatever the settings a caller hands it. This covers the `ask`, `query` and `dry_run` tools and the in-memory join of a cross-source query. Only a deployment that sets `ForceReadOnly: false` lets an Admin turn read-only enforcement off (globally or per project), and only there does the setting reach the gate. The host logs a warning at startup for each of these two locks that is off. - A **ceiling** clamps a numeric value downward only. It can lower what an administrator configured, never raise it. Clamped fields are flagged in the UI. `MaxRowLimit` bounds the rows the tools return; `StatementTimeoutSeconds`, `MaxResultBytes`, `MaxExplainCost` and `MaxConcurrentQueriesPerKey` clamp the settings of the same names. - `Ceilings:MaxSqlChars` (default `100000`) and `Ceilings:MaxQuestionChars` (default `4000`) cap the length of SQL and of `ask` questions, and `Ceilings:AskTimeoutSeconds` (default `120`, 1 to 3600) and `Ceilings:AskMaxLlmCalls` (default `12`, at least 2: the intent classification and the first SQL generation) cap the wall-clock time and the LLM calls of one `ask` call (see [`ask`](#ask)), and `Ceilings:AskSchemaContextChars` (default `120000`, about 30,000 tokens) caps the schema context one `ask` sends to the LLM: the first generation, every self-consistency candidate and every repair get the same bounded context. Over the cap, whole table descriptions are left out, never cut mid-table: the tables the question names as whole words stay first (only the one in that schema when the question writes `schema.table`), then the tables retrieval ranked for the question, then the rest in catalogue order; a table whose description does not fit is skipped and a shorter one after it may still be shown. One line `-- N more tables not shown: …` names the tables left out (the list itself capped near 2,000 characters) and replaces the coverage note; it is counted inside the cap. When the grounding sections leave no room for the most relevant table they are left out one kind at a time — learned patterns, then the glossary, the verified examples, the value matches and last the join paths — until it fits, a table is never sent whole when it does not fit: if no whole table fits, the most relevant one is cut at the last line that fits and followed by `-- name: more columns not shown` (a warning with counts only is logged), so the context is never longer than the cap and never describes no table when the source has one. A repair over SQL that reads a table left out gets that table's exact schema, however the SQL spells its name (quoted, unqualified or with a database in front), in the single-source `ask` and in each source of a cross-source `ask`. The `get_query_context` tool and cross-source `ask` read the same bounded context; a cross-source `ask` bounds each source's context on its own, so the join step's prompts can carry up to the cap once per source. They **always apply** at their defaults, even when the section is absent; the length caps must be greater than 0 when set, and the context cap at least 10000. With no section, read-only enforcement and PII detection are locked on, content retention and row data to the LLM are not locked, there are no deployment-set ceilings, and these length caps and `ask` budgets hold. - Settings resolve as **lock → project override → global value → built-in default**, then ceilings are applied. - `GET /beacon/api/mcp/settings` returns the **resolved** settings (what the tools read, locks and ceilings applied) as before, plus `stored` (the row as saved, before locks and ceilings), `lockedFields` and `clampedFields` (the settings a ceiling lowers). The global page edits `stored`, so a save keeps a value above a ceiling and the value a lock overrides: lifting the lock or raising the ceiling restores what was configured underneath. `PUT` stores every setting as sent, except a locked one, which it keeps as stored (and refuses with `409` when the value sent contradicts the lock). An API client that sends the resolved values back stores them: send `stored` instead. - **Every change is recorded.** Each setting a save changes, globally or for a project, is written to the settings history the [Admin Settings](https://moberghr.github.io/beacon/security/admin-settings/) page keeps: key `Mcp.{Setting}` or `Mcp.Project.{projectId}.{Setting}`, the value before and after (an empty override is "inherit"; a long prompt or description is cut to 2,000 characters), when, and who (the caller's Beacon user id). The log notes the change by setting name and ids only. :::caution[Upgrading] Read-only enforcement and PII detection can no longer be turned off at runtime unless the deployment unlocks them. A deployment that had either switched off globally or for a project runs with it on after the upgrade; the stored value is kept, and takes effect again only if the host sets `Beacon:Mcp:ForceReadOnly` (or `ForcePiiDetection`) to `false`. The host logs one warning after startup listing the stored rows a lock overrides (the global row and project ids), and a statement refused only because the lock turned read-only enforcement on names `Beacon:Mcp:ForceReadOnly` in its reason. ::: ### What reaches the LLM provider Beacon sends prompts to the LLM provider configured in [Admin Settings](https://moberghr.github.io/beacon/security/admin-settings/). Besides the question, the schema (table and column names, types, descriptions, join paths) and your curated knowledge (glossary, documentation, verified examples, learned patterns), a prompt can carry data read from your data sources. Each kind is off by default or screened, and each is logged: | Row data (egress category) | When | Screened | |---|---|---| | Sample values (`sample-values`, `Examples: [...]`) in `ask` and repair prompts, and returned by `get_query_context` | sample-value collection on (off by default) in the effective settings of the project and of every project sharing the source | name and value rules of [Sample values](https://moberghr.github.io/beacon/features/data-sources/#sample-values), again at every read | | Value matches (`value-matches`, `## Value matches`) for literal-looking words of the question | value grounding on (off by default), same sharing rule; live probes only for callers who may query | PII, secret-like and financial-amount column names are never probed or matched; a PII- or credential-shaped hit is dropped | | Distinct column values in documentation prompts (`column-values`, enum discovery) | sample-value collection on for the project and every project sharing the source | the same name and value rules | | Result rows in the eval judge's prompt (`result-rows`) | the eval judge on (off by default) and content retention on | PII columns masked, at most 20 rows | | Database errors in repair prompts and in lesson extraction (`database-errors` for lesson extraction) | an `ask` or cross-source `ask` repair after a failed dry run or execution; the learning job mining failed `ask` signals | redacted by class ([Database errors](#database-errors)); under the lock repair prompts and lesson extraction send only the class and code | | Stored documentation of the business-domain, data-model and glossary sections (`stored-documentation`), which can list values discovered in the data | knowledge answers: the documentation, its chunks and the project search results cut from these sections (keyword and semantic); contextual-retrieval blurbs of documentation chunks; a documentation instruction on such a section | under the lock these sections, their chunks and their search results are left out of knowledge answers and blurbs, and an instruction on one is refused (`400`); regenerate the documentation instead | - **The deployment lock `Beacon:Mcp:ForbidRowDataToLlm`** (off by default) pins sample-value collection, value grounding and the eval judge off for the global settings and every project: no sample is collected or shown, no value is probed, documentation discovers no column values, the judge sees no rows, repair prompts and lesson extraction send only an error's class and code, and stored documentation sections that can list discovered values stay out of every prompt. A request that would turn one of the three settings on answers `409`, and a project override of one is refused even when it agrees with the lock; a save under the lock keeps the values stored before it (globally and per project), so lifting the lock restores them. After startup the host logs one warning listing the stored rows the lock overrides (global row and project ids), and one naming the projects whose stored documentation still has value-listing sections; values discovered before the lock stay in that documentation until it is regenerated (a curated section, one a person edited, is kept by regeneration: edit it to remove them). - **Egress log.** Every request whose prompt carries row data is logged at Information with event id `7123` (`RowDataSentToLlm`) as it is handed to the provider. The data may leave even when the provider then fails, so a failure adds a Warning under the same event with the failure's type and code. Each record names the categories, the data source ids, the project id and the caller: the caller's Beacon user id, or for background work the job itself (`learning-aggregation`, `documentation-indexing`, `pattern-replay`; an eval run names the user who started it, else `eval-run`; documentation generation the user who asked for it), and `unknown` when the caller cannot be resolved (which never fails the request). A `get_query_context` response carrying sample values or value matches is recorded under the same event as returned to the MCP client. Never a value or prompt text; no table is written. See [MCP observability](https://moberghr.github.io/beacon/mcp/observability-and-audit/#row-data-sent-to-the-llm-provider). - **Where the data goes.** OpenAI and Anthropic are called on their public API endpoints; there is no base-URL or proxy setting for them. Azure OpenAI takes your resource's endpoint (choose its region for residency) and Bedrock your AWS region, both in the LLM connection settings. ### Database errors A database quotes row values back in its errors. A failed conversion names the value it could not convert, and a key violation names the key. Beacon therefore judges an error by its **class** (SQLSTATE, error number or the engine's wording) before it leaves its hands: - An error that **carries values** is replaced by its class, code and a generic description, with none of its message kept. Such errors are a data exception or an integrity violation (SQLSTATE `22xxx`, `23xxx`), SQL Server's conversion, overflow, duplicate-key, truncation and NULL errors, MySQL's incorrect-value, duplicate-entry, out-of-range and too-long errors, and BigQuery, Databricks and Snowflake cast and parse errors. For example: `Database error 22P02 (data exception): a value could not be converted to integer. The database's message is withheld because it can quote row values.` - An error that **carries no values** (an unknown or ambiguous name, a grouping, syntax, type, permission or division-by-zero error) keeps its message with quoted text, key and row details and unquoted values replaced by ``. Quoted text survives only in an error about a name, and only when every part of it is a name of the statement or the catalog. - An error Beacon does not recognise is replaced like a value-carrying one, whatever it says, quoted or not: `Database error (unrecognised): not an error Beacon recognises. The database's message is withheld because it can quote row values.` So is an error the statement's own code raised, whose wording is its author's: a SQLSTATE of class `P0` (PostgreSQL's `RAISE`, Databricks' `raise_error`), SQL Server's errors 50000 and above (`THROW`, `RAISERROR`) and MySQL's SQLSTATE class `45` (`SIGNAL`) keep only their code, even when the text reads like a name, syntax or permission error. Where the driver leaves the code out of the message, Beacon puts it in front (`Msg 50000, Level 16, State 1: …`, `45000: …`), also in the raw message a caller reads while PII detection is off. - Beacon's own fixed texts that arrive with the database's errors (a host source's failure, a read stopped at the row ceiling, a refused read of a migration write target) hold no value and are kept. The whole message is redacted before it is cut to 2,000 characters; an error that cannot be screened in time is replaced as a whole and counted (`beacon.error_redaction.fallbacks`). Where the redacted form is used: | Where the error goes | Redacted | |---|---| | Repair prompts (`ask`, cross-source `ask`) and lesson extraction | always; under the lock `Beacon:Mcp:ForbidRowDataToLlm` only the class and code (`Database error 42703 (undefined column).`) | | Learning signals (`ask`, `query`, `dry_run`, cross-source) and the stored history of ad-hoc queries | always | | The error returned by `ask`, `query`, `dry_run`, cross-source queries, saved-query tools and REST previews and executions, and the audit row | while PII detection is on for the project (or the strictest project sharing the source); the raw message otherwise | | Host data sources | never the database's message: `Query failed on the host database.` | Logs never carry the message at all: a failed query is logged by exception type, error code, the types of the exceptions it wraps and the method that threw it (`PostgresException (error code 22P02, inner SocketException, at NpgsqlConnector.ReadMessage)`). :::caution[Upgrading] - **Defaults.** A new installation starts with sample-value collection and value grounding off. Stored settings are not changed: an installation that saved its global MCP settings keeps the values it stored (the migration `DefaultSampleValueCollectionOff` only changes the column default for a row created later). An installation that never saved them reads the new defaults, so both are off until a project or the global settings turn them on; the host then logs at startup which row-data settings are in effect. With sample-value collection off, documentation generation lists no discovered column values (logged once per run). - **Sampling, in both directions.** A project that turns sample-value collection on now collects even while the global value is off (before, the global value had to be on too). A refresh no longer reads excluded columns at all, and more names are excluded: person names, Croatian personal data, connection strings, `dsn`, webhook URLs, `hmac` and key material named with `hmac`, `aes`, `rsa`, `pgp`, `gpg` or `ssh` beside `key`. Surrogate keys such as `ProductKey` are no longer excluded and are sampled again. The next refresh replaces what changed. - **Settings API.** A `PUT /beacon/api/mcp/settings` body that omits `enableSampleValueCollection` or `enableValueGrounding` now stores `false` (the default) for it; send every field you read. The settings page shows both switches and sends back every setting it loaded, the ones it does not show included. - **Errors.** Database errors returned to callers and written to the audit are in the generic class form above while PII detection is on, and learning signals and ad-hoc query history always hold the redacted form. An error of no class Beacon recognises is now withheld even without a quote in it, and an error raised by the statement's own code (SQL Server 50000 and above, MySQL SQLSTATE class 45) reaches a caller led by its code. Failure logs name the failure as `Type (error code …, inner …, at …)` instead of separate type and code fields. - **Masking.** Banking identifiers (`iban`, `oib`, `jmbg` as whole name segments, `account_number`) are PII names: query results mask such columns on every path while PII detection is on. Data-quality custom SQL masks a text value (an `actual_value` or `message`) that looks like PII, whatever its column is called. - **Under the lock**, an instruction on a business-domain, data-model or glossary documentation section answers `400`. ::: The per-project settings have their own endpoints: `GET /beacon/api/mcp/projects/{projectId}/settings` (Auditor or SecurityAdmin) and `PUT` on the same route (SecurityAdmin). The `GET` returns the project's overrides, the resolved effective values, and the names of any locked or clamped fields. Changes take effect without a restart. These settings are part of [Admin Settings](https://moberghr.github.io/beacon/security/admin-settings/); the retrieval, exemplar, and eval knobs that aren't on this page are settable through `PUT /beacon/api/mcp/settings` and documented in the [Knowledge Base guide](https://moberghr.github.io/beacon/mcp/knowledge-base/#settings-reference). ## MCP Playground You do not need to wire up an external MCP client to try the server. The UI ships an **MCP Playground** (`/beacon/mcp-playground`) where you select a project and ask questions interactively. It uses the same tools, routing, guardrails, and audit trail as a real MCP session. Use it to validate documentation quality and tune the system prompt before pointing Claude or Cursor at your data. The playground runs as the signed-in user and checks their [project access](https://moberghr.github.io/beacon/security/authorization/#project-access) on every call: the catalog tools (`get_context`, `get_documentation`, `search`, `get_query_context`) need Read in the selected project, and `ask`, `query`, `dry_run` and `feedback` need Query. A restricted project the user is not a member of, or a project outside an API key's list, is refused before any tool runs; a call without a signed-in request is refused outright. Tool calls over `/beacon/mcp` decide the caller's projects by the same rules (see [Overview](#overview)). The companion **MCP Learning** page (`/beacon/mcp-learning`) shows the learning loop at work: usage signals, success rate, and learned schema patterns awaiting approval. Any stored documentation patch can also be applied or rejected there. ## Calling the endpoint directly A client without native Streamable HTTP support can POST JSON-RPC messages to `/beacon/mcp` with the `Authorization: Bearer` header on every request. The request shapes, headers and session handling are in [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/). ## Troubleshooting **"No projects are available to you" error**: the caller reaches no project: none of the API key's projects is one its owner may still use, a mapped Entra user is not a member of the restricted projects it is configured for, or the call came from a browser session (use the [MCP Playground](#mcp-playground)). Check the key's projects and the owner's or user's project memberships. **"Access denied: you do not have access to project N"**. The project named in `project_id` is not one the caller reaches (see [Overview](#overview)). **"Access denied: you may read project N but not run ad-hoc queries in it"**. The caller is a Viewer in the project: it reads the catalog, but `ask`, `query` and `dry_run` need an Editor role there (and, in a restricted project, an Editor membership). Saved-query tools stay available to Viewers. **"Multiple projects available (IDs: …)"**. The caller reaches several projects. Pass `project_id` on every call, or restrict the API key to a single project. **SQL refused by the read-only gate**. The statement is not a single read-only query (an `INSERT`, `UPDATE`, DDL, a second statement), or it names another database than the data source's own (`cross_database`), or it reads the engine's catalog, statistics or query history (`system_catalog`). Run `dry_run` to see every issue with its gate and code. **Connection drops, or "session not found" behind a load balancer**: streaming connections may be interrupted by proxies, and an initialize-handshake client's session lives on one instance and closes after 30 minutes idle. Re-initialize against `/beacon/mcp` to start a fresh session, and give multi-instance deployments session affinity (see [Connection details](#connection-details)). **Authentication fails**: Verify your API key starts with `sk-sem_`, has not expired, and has not been revoked. Check the `Authorization: Bearer` header format. **"Permission denied: this credential does not have the Execute scope"**. The key (or Entra caller) has only `Read`, or it has `Execute` but its owner no longer has the Editor role or above. Use an `Execute` key of a user with write permission. **Answers miss obvious joins or misread a business term**. The model can only use what it was given. Check the [Knowledge Base guide](https://moberghr.github.io/beacon/mcp/knowledge-base/): register the join path, add the glossary term, or promote a correct answer to a golden example. ## Upgrading from 4.5: project access MCP now uses the same [project access](https://moberghr.github.io/beacon/security/authorization/#project-access) as the REST API and the playground, decided on every call (see [Overview](#overview) and the full [upgrade notes](https://moberghr.github.io/beacon/security/authorization/#upgrading-from-45-project-access)): - **API keys without a project list** now reach their owner's projects on MCP; before, they reached no project there. This widens what such keys can do over MCP. Re-issue them with a project list to confine them. Keys with a list reach only the listed projects their owner may still use. - **Mapped Entra user callers** reach a restricted project only while their Beacon user is a member of it, and nothing while their user is disabled or archived; a user caller without a Beacon user reaches its configured open projects only. System callers keep their configured projects. See [Entra ID callers on MCP](https://moberghr.github.io/beacon/mcp/entra-callers/). - **Saved-query tools** are listed only in projects where the caller may run saved SQL, and each run is re-checked right before its SQL: when a data source the version reads is no longer linked to the project, or the caller may no longer run saved SQL there, the call fails without running anything. **Host endpoint tools** need the caller to read the host's project. - **A call without an authenticated caller is refused**, over HTTP or over a transport without an HTTP request (such as a host-wired stdio transport), and such a call reaches no project. Browser sessions are refused on `/beacon/mcp`: the route and the MCP layer take an API key or a mapped bearer token only. A refusal at the route answers `403` before any MCP request is read, so it leaves no audit row; a `tools/call` refused inside the MCP layer is audited. - **The SQL-carrying tools need Query in the project**, as in the playground: `ask`, `query` and `dry_run` (and `feedback`) answer "Access denied: you may read project N but not run ad-hoc queries in it." to a caller that is only a Viewer there, while the catalog tools still answer it. Mapped or provisioned Entra users keep the default `Viewer` role (`Beacon:Mcp:Callers:Users:DefaultRoleName`) and so read the catalog only: give them the Editor role (and, in a restricted project, an Editor membership) to run SQL tools. API keys are already capped by their owner's role. - **Error messages** no longer mention API keys: "Access denied: you do not have access to project N." and "No projects are available to you." - **Audit rows and signals** record the caller's Beacon user id. API keys and mapped callers were already recorded so; a signed-in session (the playground) was recorded by its external id, with no user when that id is not a number, so it could not rate the signals of its own `ask` calls. Now it can. - **Eval runs without a project** are refused: the run is marked `Failed` before any case runs. - **Database names must be the data source's own.** SQL naming another database, catalog or project than the one the data source's connection configures is refused (`cross_database`), and a data source without a configured database accepts no database qualifier. SQL the gate cannot parse is refused even with read-only enforcement off. - **The engine's catalog is not queryable.** SQL reading `information_schema`, `pg_catalog`, `sys` or another catalog, statistics or query-history relation is refused (`system_catalog`); use `get_context` for tables and columns. - **Shared data sources take the strictest guardrails** of the live projects that link them: a project with a looser row limit, PII, read-only, value-grounding or sample-value setting gets the stricter one on a data source it shares. - **Sample values and value grounding are off by default** for a new installation, and the deployment lock `Beacon:Mcp:ForbidRowDataToLlm` keeps all row data from the LLM provider; see [What reaches the LLM provider](#what-reaches-the-llm-provider). - **`get_query_context` runs live value probes only for callers who may query the project**; a Viewer's context resolves values from the stored samples alone. ## See also - [Knowledge Base & Grounding](https://moberghr.github.io/beacon/mcp/knowledge-base/): what grounds generated SQL, and how to curate it - [API Keys](https://moberghr.github.io/beacon/security/api-keys/): scopes, project restrictions, and rotation - [Entra ID callers on MCP](https://moberghr.github.io/beacon/mcp/entra-callers/): user and system JWT callers, their projects, scope and audit - [Admin Settings](https://moberghr.github.io/beacon/security/admin-settings/): LLM provider configuration - [AI Integration](https://moberghr.github.io/beacon/features/ai-integration/): documentation generation that feeds the MCP catalog tools --- # Connect AI clients > Expose Beacon's MCP server from your host, decide how AI clients authenticate (API keys or Entra ID callers), publish discovery metadata, and connect Claude Code, Claude, Cursor and VS Code. Source: https://moberghr.github.io/beacon/mcp/connect/ Beacon's MCP server lets AI assistants explore and query your data through eight tools, always project-scoped, read-only, masked and audited. This page wires it into your host and connects clients. The tools themselves are described on [MCP server](https://moberghr.github.io/beacon/mcp/server/). ## Wiring ```csharp builder.Services.AddBeaconApiAuthorization(); // registers the Execute-scope policy MCP needs builder.Services.AddBeaconAI(builder.Configuration); builder.Services.AddBeaconMcp(); // … app.UseMcpResourceMetadataChallenge(); // before UseBeaconAuthentication app.UseBeaconAuthentication(); // API keys, JWT callers (when enabled), the cookie app.UseAuthorization(); app.MapBeaconMcp(); // /beacon/mcp, Execute scope required app.MapMcpDiscovery(); // anonymous discovery documents ``` | Item | Value | |---|---| | Endpoint | `/beacon/mcp` | | Transport | Streamable HTTP | | Authentication | `Authorization: Bearer sk-sem_…` (API key) or `Authorization: Bearer ` | | Required scope | **Execute** (a Read-scoped key gets 403) | | Built-in tools | All eight by default; `AddBeaconMcp(x => …)` chooses (`BuiltInTools`), turns saved-query tools off (`SavedQueryTools = false`) and replaces the server instructions (`ServerInstructions`) and the server card's description (`ServerDescription`). A tool left out is neither listed nor callable, and the server card lists only the ones served. | | Sessions | Clients that start with `initialize` (most clients today) get an `Mcp-Session-Id` (idle timeout 30 minutes) and need **session affinity** when you run several instances. Clients speaking the stateless protocol revision `2026-07-28` need no session. A request that is neither (no session and no `2026-07-28` metadata) is refused with 400. | `MapBeaconMcp()` maps the route with the Execute-scope policy, and two filters inside the MCP layer refuse any caller without Execute even if the route were mapped with a weaker policy. Do not map it yourself with `MapMcp(...)` and a bare `RequireAuthorization()`. :::note[Browser sessions] A signed-in browser session, Beacon's or your host's, is refused on `/beacon/mcp` with `403`: the route takes an API key or a mapped bearer token only. People try the tools in the UI instead: **MCP playground** (`/beacon/mcp-playground`). ::: ## Public URL and discovery Clients that connect by URL (claude.ai, Claude Desktop connectors, VS Code) discover how to authenticate from three anonymous documents: | Path | Content | |---|---| | `/.well-known/oauth-protected-resource` | RFC 9728 protected-resource metadata: `resource` (`{base}/beacon/mcp`), `scopes_supported`, `authorization_servers`, bearer in header only | | `/.well-known/oauth-protected-resource/beacon/mcp` | The same, at the path-inserted location | | `/.well-known/mcp/server-card.json` | Server card: name, transport, endpoint, authentication, the eight built-in tools | Every 401 from `/beacon/mcp` carries `WWW-Authenticate: Bearer resource_metadata="{base}/.well-known/oauth-protected-resource"`. **Set `Beacon:PublicBaseUrl`** (for example `https://beacon.example.com`) in every proxied or public deployment. Without it, the documents and challenges are built from the request's `Host` header, which the client controls; Beacon warns at startup. To point OAuth-capable clients at your identity provider: ```json { "Beacon": { "PublicBaseUrl": "https://beacon.example.com", "Mcp": { "ProtectedResource": { "AuthorizationServers": [ "https://login.microsoftonline.com//v2.0" ], "ScopesSupported": [ "api:///mcp.access" ] } } } } ``` | Key | Default | What it does | |---|---|---| | `AuthorizationServers` | None | Issuers published as `authorization_servers` (each must equal the server metadata's `issuer`). Takes precedence. | | `DeriveAuthorizationServerFromJwtIssuer` | `false` | Publish the bearer-JWT issuer instead (prefers an Entra `/v2.0` issuer). | | `ScopesSupported` | `["Execute"]` | Scopes published to clients. Set your identity provider's scope when you publish an external authorization server, and add `offline_access` so clients get a refresh token instead of asking people to sign in every hour (Beacon warns at startup when an `api://` scope is published without it). A configured list replaces the default whole. | | `ResourceName` | `Beacon MCP` | The `resource_name` an MCP client shows for the server, also the server card's `name` and the server name sent at `initialize` (`Beacon` when empty). A host serving Beacon as its own MCP server names its application. | | `ResourceDocumentation` | Beacon's MCP docs | The `resource_documentation` URL, also the server card's `documentation`. Anything but an absolute https URL logs a startup warning. | Code can complete these settings too: `services.PostConfigure(...)` runs after the configuration is bound, on every request. With none of these set and OIDC enabled, Beacon publishes its OIDC authority. ## Option A: API keys The simplest path: each person creates a key for themselves. 1. In **API keys** (`/beacon/api-keys`), create a key with the **Execute** scope, limited to the projects the assistant should use, with a short expiry. (Scripts can do the same through `POST /beacon/api/api-keys`; see [headless first run](https://moberghr.github.io/beacon/integrate/authentication/#headless-first-run-scripts-and-ci).) 2. Copy it. It is shown once. 3. Configure the client with `Authorization: Bearer sk-sem_…` (below). The key acts as its owner, only in the key's projects, capped by the owner's current role: reading tools and saved-query tools work for any owner, while `query` and `ask` need an owner who can query (Editor or above). When a key reaches several projects, the assistant passes `project_id` on each call. ## Option B: Entra ID callers For organisation-wide access without personal keys, Beacon accepts Entra access tokens on `/beacon/mcp` and maps each token to an **MCP caller** with projects and a scope. Two kinds: - **Users**: delegated (on-behalf-of) tokens from a client app your users sign in to; optionally auto-provisioned as Beacon users. - **Systems**: app-only tokens of a known application or service principal. ```csharp builder.Services.AddBeaconJwtAuthentication(jwt => { jwt.EnableBearerAuthentication = true; jwt.Validation.JwksEndpoint = "https://login.microsoftonline.com//discovery/v2.0/keys"; jwt.Validation.ValidIssuers = [ "https://login.microsoftonline.com//v2.0" ]; jwt.Validation.ValidAudiences = [ "api://" ]; }); ``` ```json { "Beacon": { "Mcp": { "Callers": { "AllowedTenants": [ "" ], "Users": { "Enabled": true, "RequiredGroups": [ "" ], "ProjectIds": [ 4 ], "GroupProjects": { "": [ 7 ] }, "DefaultRoleName": "Viewer" }, "Systems": [ { "Name": "reporting-bot", "ClientId": "", "ProjectIds": [ 4 ], "Scope": "Execute" } ] } } } } ``` | Key | Default | What it does | |---|---|---| | `AllowedTenants` | Required | Tenants whose tokens may map to a caller. Required once any caller is configured. | | `Users:Enabled` | `false` | Accept delegated user tokens. | | `Users:RequiredRoles` / `RequiredGroups` | None | The user needs one of them; with neither set, `AllowAnyTenantUser` must be `true`. | | `Users:ProjectIds`, `Users:GroupProjects` | None | Projects every accepted user gets, plus extra projects per group or app role. | | `Users:Scope` | `Execute` | `Read` or `Execute`. | | `Users:AutoProvision` / `DefaultRoleName` | `true` / `Viewer` | Create a Beacon user on first call, with this role (never Admin or a functional role). | | `Systems[]` | None | `Name`, exactly one of `ClientId` (app-only token) or `ObjectId` (service principal), `ProjectIds`, `Scope`, `HostClaims`. | An absent `Callers` section admits no JWT caller on MCP; it fails closed. Startup validates the section, including that bearer JWT validation is on with explicit issuers and audiences. Audit rows identify Entra callers by a keyed hash (`Beacon:Mcp:CallerHashKey`, else derived from the encryption key). Full reference: [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/). ## Connect a client ### Claude Code ```bash claude mcp add --transport http beacon https://beacon.example.com/beacon/mcp \ --header "Authorization: Bearer $BEACON_MCP_API_KEY" ``` Or share it with your team in the repository's `.mcp.json` (Claude Code expands environment variables): ```json { "mcpServers": { "beacon": { "type": "http", "url": "https://beacon.example.com/beacon/mcp", "headers": { "Authorization": "Bearer ${BEACON_MCP_API_KEY}" } } } } ``` ### Cursor `~/.cursor/mcp.json` or `.cursor/mcp.json` in the project: ```json { "mcpServers": { "beacon": { "url": "https://beacon.example.com/beacon/mcp", "headers": { "Authorization": "Bearer sk-sem_…" } } } } ``` ### VS Code `.vscode/mcp.json`: ```json { "inputs": [ { "type": "promptString", "id": "beacon-key", "description": "Beacon API key", "password": true } ], "servers": { "beacon": { "type": "http", "url": "https://beacon.example.com/beacon/mcp", "headers": { "Authorization": "Bearer ${input:beacon-key}" } } } } ``` ### Claude (claude.ai and Claude Desktop) Add Beacon as a **custom connector** with the URL `https://beacon.example.com/beacon/mcp`. Connectors authenticate with OAuth, so this path needs Option B: publish your Entra authorization server and scope through `Beacon:Mcp:ProtectedResource`, register the client's redirect URI in your Entra app, and map users with `Beacon:Mcp:Callers:Users`. The step-by-step Entra setup is on [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/). Client configuration formats change; if a snippet does not work, check the client's own documentation. The endpoint, header and transport above are what Beacon needs. ## Check it works ```bash BEACON=https://beacon.example.com curl -s $BEACON/.well-known/mcp/server-card.json | jq '.tools[].name' H=(-H "Authorization: Bearer $BEACON_MCP_API_KEY" -H "Content-Type: application/json" -H "Accept: application/json, text/event-stream") # 1. initialize: the response header Mcp-Session-Id names the session SESSION=$(curl -s -D - -o /dev/null "${H[@]}" $BEACON/beacon/mcp \ -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"1"}}}' \ | awk -F': ' 'tolower($1)=="mcp-session-id"{print $2}' | tr -d '\r') # 2. confirm initialisation curl -s "${H[@]}" -H "Mcp-Session-Id: $SESSION" -H "MCP-Protocol-Version: 2025-06-18" $BEACON/beacon/mcp \ -d '{"jsonrpc":"2.0","method":"notifications/initialized"}' # 3. list the tools curl -s "${H[@]}" -H "Mcp-Session-Id: $SESSION" -H "MCP-Protocol-Version: 2025-06-18" $BEACON/beacon/mcp \ -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' ``` A client on the stateless revision skips the session: it sends `MCP-Protocol-Version: 2026-07-28` and `Mcp-Method: tools/list` headers (plus `Mcp-Name: ` on `tools/call`) and puts `{"io.modelcontextprotocol/protocolVersion":"2026-07-28","io.modelcontextprotocol/clientCapabilities":{}}` in `params._meta`. Then ask the assistant something like *"Use Beacon's get_context to describe my project."* | Symptom | Cause | |---|---| | 400 "A new session can only be created by an initialize request" | The client sent a request without a session: start with `initialize` (or use the stateless `2026-07-28` revision). | | 401 with `resource_metadata` | No or invalid credential. | | 403 at the door | The key has the Read scope, the JWT caller is not mapped, or a browser session is calling. | | "Permission denied: this credential does not have the Execute scope" | Same, reported as a tool error. | | "No projects are available to you." | The key or caller reaches no project. | | "Access denied: you may read project N but not run ad-hoc queries in it." | Viewer-level access: `query`/`ask` need Query (Editor). | ## Add your own tools - Publish an approved saved query as an MCP tool `q_` without code: [Saved query tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/). - Expose endpoints of your own API as `api_` tools: [Extending Beacon](https://moberghr.github.io/beacon/integrate/extending/#your-endpoints-as-mcp-tools). ## Governing MCP Per-project MCP settings (row limits, read-only, PII detection, content retention, learning) live on **MCP settings** (`/beacon/mcp-settings`), editable by a SecurityAdmin and readable by an Auditor. Deployment locks (`Beacon:Mcp:ForceReadOnly`, `ForcePiiDetection`, `ForceNoContentRetention`, `ForbidRowDataToLlm`) and ceilings (`Beacon:Mcp:Ceilings`) bind them from configuration. Every call is audited; see [MCP observability & audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/). --- # Entra ID callers on MCP > Let an agent platform call Beacon's MCP endpoint with Microsoft Entra ID tokens, either on behalf of a user or as a system identity, with projects and scope decided by Beacon configuration. Source: https://moberghr.github.io/beacon/mcp/entra-callers/ Besides [API keys](https://moberghr.github.io/beacon/security/api-keys/), the [MCP server](https://moberghr.github.io/beacon/mcp/server/) at `/beacon/mcp` accepts **Microsoft Entra ID bearer tokens**. This is how an agent platform calls a Beacon that is embedded in a host application. It supports two modes, and both are first-class: | Mode | Token | Identified by | |------|-------|---------------| | **User** | A delegated token issued on behalf of the user (it carries `scp`) | The user's `oid`, plus `roles` and `groups` | | **System** | An app-only token from the client-credentials flow (`idtyp=app`, or no `scp` + `roles` + `oid == sub`) | The calling app's `azp` / `appid` | | **System** (service user) | A token for a dedicated Entra service user | The service user's `oid` | The token only proves **who** is calling. **What** the caller may do (which projects, and which scope) comes from Beacon configuration. A token cannot grant itself projects or a scope: any `allowed_projects`, `scope`, `auth_method`, `api_key_id` or `caller_*` claim inside the token is discarded. ## How a call is resolved 1. The bearer token is validated (signature against the JWKS endpoint, issuer, audience and lifetime). The JWKS is cached: it is fetched once and refreshed hourly, or earlier (at most once every 5 minutes) when a token is signed with a key id the cached set does not hold (key rotation). A failed refresh keeps the last good keys for at most 24 hours after the last successful fetch; after that, bearer requests answer `503` until a fetch succeeds. 2. On `/beacon/mcp`, Beacon passes the validated claims to `IMcpCallerMapper`. The default `ConfiguredMcpCallerMapper` first rejects the token unless: - its `tid` is one of `AllowedTenants` (a token without `tid` is rejected); - its `aud` is one of the bearer audiences Beacon validates (checked again here, as defense in depth); - it is an **access token** of a known kind (see [Token classification](#token-classification)); ID tokens and tokens that cannot be classified are rejected. It then checks, in this order: 1. A configured **system** whose `ObjectId` equals the token's `oid` (a service user or a service principal). 2. A configured **system** whose `ClientId` equals `azp`/`appid`, but **only for app-only tokens**. A delegated user token (or an ID token) issued to the same app also carries its `azp`, so it never matches a `ClientId` system. 3. **User mode**, for delegated tokens, when it is enabled and the caller has one of the required roles or groups (or `AllowAnyTenantUser` is set). 3. A matched caller gets the Beacon claims that API keys already use: `allowed_projects`, `scope` (`Read` or `Execute`), and the Beacon user id as the name identifier when the user is provisioned. It also gets `caller_kind` and `caller_hash`. 4. The route's Execute-scope policy then applies to JWT callers exactly as it does to API keys. **`Read` callers and unknown callers get `403`**, and `Execute` callers pass. A caller that is neither a configured system nor an accepted user is **unknown**. It gets no projects and no scope, and it is rejected. This fails closed. The mapped principal carries `auth_method=mcp_caller`, so the Execute-scope rules for scoped callers apply to it as they do to API keys. A browser session (cookie or SSO, Beacon's or the host's) is not a scoped caller and is refused on `/beacon/mcp` with `403`; browsers use the [MCP Playground](https://moberghr.github.io/beacon/mcp/server/#mcp-playground). ### What a caller reaches The configured projects are what a caller is mapped to. What it reaches in them is decided on every call by Beacon's [project access](https://moberghr.github.io/beacon/security/authorization/#project-access), as for API keys and signed-in users: | Caller | Reaches | |--------|---------| | System | its configured projects, restricted or not | | User with a Beacon user (`AutoProvision`) | its configured projects within its Beacon user's access: open projects, and a restricted project only while the user is a member of it, at the lower of its membership role and its Beacon roles | | User with a disabled or archived Beacon user | nothing | | User without a Beacon user (user management off, or `AutoProvision` off) | its configured **open** projects only | A membership granted or revoked, a project restricted, or a user disabled takes effect on the caller's next call. Provisioned users get no membership automatically: add them as members of the restricted projects they are configured for (`DefaultRoleName` gives them their global role, `Viewer` by default). What a user caller with a Beacon user may **do** in a project follows that user's role there, capped by the caller's scope: the catalog tools (`get_context`, `search`, `get_documentation`, `get_query_context`) need Read; `ask`, `query`, `dry_run` and `feedback` work ad hoc and need an Editor role there: the user's global role Editor or above and, in a restricted project, an Editor membership (or project admin). A user provisioned with the default `Viewer` role therefore reads the catalog and runs saved-query tools, but its SQL tools answer "Access denied: you may read project N but not run ad-hoc queries in it." (audited). Raise `DefaultRoleName`, or the user's role, to let it run SQL. System callers, and user callers without a Beacon user, are capped by their scope only (`Execute`: SQL tools allowed). Provisioning therefore changes what the same person can do. A user caller **without** a Beacon user acts within its configured open projects with the SQL tools (its `Execute` scope allows them); once provisioned, it gets its stored role instead. With the default `Viewer`, this is the catalog only. Both are in the administrator's hands: the caller configuration (its projects and scope) decides the first, `DefaultRoleName` and the user's roles the second. When user callers are configured and `DefaultRoleName` is `Viewer`, Beacon logs this once at startup (Information). Other routes (`/beacon/api/*`) accept a bearer token only when it names an **existing, enabled, external Beacon user** by its subject (`sub`) and issuer, or a pre-registered user's subject when only one issuer is configured. A user that MCP user provisioning created (keyed on `oid` within the tenant) gets **no** REST session from a token. The session is that user, with the user's **Beacon roles**; no token claim (roles included) reaches it. Any other token gets `401` with `WWW-Authenticate: Bearer error="invalid_token"`. A configured system identity therefore reaches only `/beacon/mcp`, where its projects and scope apply. On `/beacon/mcp` the principal carries only the caller's name, e-mail and display name from the token, plus what the mapper decides: token roles, groups, `beacon:*` claims and the token's own subject never reach it. Until first-run setup has created the super admin, `AutoProvision` creates no user and such a caller is rejected. ### Token classification "No `scp`" alone does not make a token app-only: an ID token, or any other token without `scp`, carries the requesting app's `azp` too and would otherwise be taken for that app's system identity. The mapper therefore needs a positive signal: | Token | Classified as | |-------|---------------| | Has `nonce` | ID token → **rejected** | | `idtyp = app` and no `scp` | App-only | | `idtyp = app` and `scp` | Contradictory → **rejected** | | Has `scp` | Delegated (user) | | No `scp`, has `roles`, and `oid == sub` (a service principal's own token) | App-only | | Anything else (e.g. neither `scp` nor `roles`) | **Rejected** | **Entra app registration requirement.** For every system that calls with the client-credentials flow, either add the optional access-token claim **`idtyp`** to Beacon's app registration (Token configuration → Add optional claim → Access → `idtyp`; recommended), or assign the calling app at least one **app role** on Beacon's app registration so its tokens carry `roles`. A client-credentials token with neither is rejected. ## Configuration Bearer JWT validation has to be on. In the sample host, turn it on by setting `Beacon:Authentication:Oidc:Enabled` and `Beacon:Authentication:Oidc:McpJwksEndpoint`; the sample then validates the issuer against the OIDC `Authority` and the audience against the OIDC `ClientId`, which may not match the v1/v2 issuer and audience pairs Entra puts in its tokens. A host that configures `Callers` usually sets `ValidIssuers` and `ValidAudiences` itself, as below. In your own host, call `AddBeaconJwtAuthentication(x => { x.EnableBearerAuthentication = true; ... })` and `app.UseBeaconJwtBearerAuthentication()`. The token audience must be Beacon's app registration. **Issuer and audience are mandatory once any caller is configured.** Entra's JWKS signs tokens for every tenant and every resource, so a valid signature proves nothing about who the token is for. When `Users:Enabled` is on or any `Systems` entry exists, the host refuses to start unless bearer JWT authentication is enabled, `Validation.ValidIssuer` (or `ValidIssuers`) and `Validation.ValidAudience` (or `ValidAudiences`) are set, and `ValidateIssuer` / `ValidateAudience` are left on: ```csharp builder.Services.AddBeaconJwtAuthentication(x => { x.EnableBearerAuthentication = true; x.Validation.JwksEndpoint = "https://login.microsoftonline.com//discovery/v2.0/keys"; x.Validation.ValidIssuers = [ "https://login.microsoftonline.com//v2.0", // v2 access tokens "https://sts.windows.net//" // v1 access tokens ]; x.Validation.ValidAudiences = ["api://", ""]; }); ``` The callers are configured under `Beacon:Mcp:Callers`: ```jsonc { "Beacon": { "Mcp": { "Callers": { "AllowedTenants": [""], // required once any caller is configured "Users": { "Enabled": true, "RequiredRoles": ["Beacon.Mcp.User"], // any-of, combined with RequiredGroups "RequiredGroups": [""], // empty + empty fails startup... "AllowAnyTenantUser": false, // ...unless this is set to true explicitly "ProjectIds": [1], // granted to every accepted user "GroupProjects": { // group id or app role -> more projects (additive) "": [2, 3], "Beacon.Finance": [4] }, "Scope": "Execute", // Read | Execute "AutoProvision": true, // find or create a Beacon user keyed on oid "DefaultRoleName": "Viewer" // global role for a newly provisioned user }, "Systems": [ { "Name": "agent-platform", "ClientId": "", // app-only token (azp / appid) "ProjectIds": [1, 2], "Scope": "Execute", "HostClaims": [ { "Type": "permission", "Value": "ViewReports" } ] }, { "Name": "routine-runner", "ObjectId": "", // dedicated Entra service user (oid) "ProjectIds": [2], "Scope": "Execute" } ] } } } } ``` | Setting | Default | Meaning | |---------|---------|---------| | `AllowedTenants` | empty | Entra tenant ids (`tid`) whose tokens may be mapped. **Required and non-empty** once `Users:Enabled` is on or any system is configured | | `Users:Enabled` | `false` | Accept delegated user tokens at all | | `Users:RequiredRoles` / `RequiredGroups` | empty | Any-of across both lists. Both empty fails startup unless `AllowAnyTenantUser` is `true` | | `Users:AllowAnyTenantUser` | `false` | Explicit opt-in to admit **every** user of the allowed tenants when no role or group is required | | `Users:ProjectIds` | empty | Projects every accepted user is configured for; a restricted one is reached only by its members (see [What a caller reaches](#what-a-caller-reaches)) | | `Users:GroupProjects` | empty | Additional configured projects per group object id or app role, added together | | `Users:Scope` | `Execute` | `Read` callers cannot reach `/beacon/mcp` | | `Users:AutoProvision` | `true` | Create or find a Beacon user (`ExternalId` = `oid`, provider `entra:{tid}`) so audit rows carry a user and project memberships apply. A disabled or archived Beacon user is rejected; within the one minute a caller's user is cached, its calls reach no project instead. This needs user management to be enabled; without it the caller is accepted with no Beacon user | | `Systems[]:ClientId` / `ObjectId` | none | **Set exactly one per entry.** The host refuses to start otherwise | | `Systems[]:HostClaims` | empty | The permission set this identity gets inside the host application. It is carried on the caller for in-process API dispatch | Validation runs at startup. A system entry with both `ClientId` and `ObjectId`, or with neither, a duplicate name, or a non-positive project id, stops the host with a message that names the entry. So do, once any caller is configured: an empty `AllowedTenants`, bearer validation without issuer or audience, and user mode with no role or group requirement and no `AllowAnyTenantUser`. With user mode and `AutoProvision` on, `DefaultRoleName` must be set and must not be the Admin role or the role of a functional permission (every provisioned user would administer Beacon). With `AutoProvision`, two first requests from the same new user can race to create the Beacon user; the loser re-reads the row the winner created instead of failing. :::caution[Group overage] Entra leaves out the `groups` claim when a user belongs to more than about 200 groups (the "overage" case). For large tenants, prefer **app roles** in `RequiredRoles` and `GroupProjects`, or assign the groups to the app registration so the token only lists relevant groups. ::: ## Connect Claude (claude.ai / Claude Desktop custom connector) Claude's custom connectors connect by URL and sign the user in with OAuth. Beacon does not run an OAuth server here: Claude gets a delegated **Entra** token for the user and calls `/beacon/mcp` with it, and Beacon validates the token and maps the user like any other [user-mode caller](#how-a-call-is-resolved). Claude learns where to sign in from Beacon's [discovery documents](https://moberghr.github.io/beacon/mcp/server/#discovery-endpoints): 1. `POST /beacon/mcp` without a token answers `401` with `WWW-Authenticate: Bearer resource_metadata="https:///.well-known/oauth-protected-resource"`. 2. That document names the authorization server (`authorization_servers[0]`, the Entra v2 issuer), the scopes to request (`scopes_supported`) and the resource (`resource`, the canonical `https:///beacon/mcp` URL). 3. Claude reads Entra's metadata at `https://login.microsoftonline.com//v2.0/.well-known/openid-configuration`, signs the user in, and requests a token for those scopes, sending the resource URL as the RFC 8707 `resource` parameter. By default the document lists Beacon's internal scope name (`Execute`) and no authorization server unless Beacon's own OIDC login is on. Neither is something Entra understands. Configure it under `Beacon:Mcp:ProtectedResource`. ### 1. The API app registration This is the app registration whose tokens Beacon validates (`Validation.ValidAudiences`). In an embedded setup, usually the host's API registration. - **Manifest:** set `api.requestedAccessTokenVersion` to `2` (`accessTokenAcceptedVersion: 2` in the older AAD Graph manifest format). Do this first: Entra only lets a v2 app register an HTTPS Application ID URI. - **Expose an API → Application ID URI:** keep `api://` and **add the exact MCP URL**, `https:///beacon/mcp`, to `identifierUris`. MCP clients send that URL as the `resource` parameter, and Entra does *not* ignore it: it must match one of the app's Application ID URIs character for character (no trailing slash), or the token request fails with `AADSTS9010010`. Entra may require the host name to be a verified domain of the tenant before it accepts an HTTPS Application ID URI. - **Expose an API → Add a scope:** `mcp.access` (delegated; admins and users, or admins only). Clients request it fully qualified, `api:///mcp.access`. The scope and the `resource` must belong to the same app registration. - **App roles** (recommended): e.g. `Beacon.Mcp.User`, assigned to the users or groups who may use the connector, and listed in `Users:RequiredRoles`. The token Claude receives is a v2 access token: `iss` is `https://login.microsoftonline.com//v2.0`, `aud` is the API's **client id GUID** (not the `api://` URI and not the MCP URL), and `scp` is `mcp.access`. Keep both audience forms in `ValidAudiences` so v1 callers of the same API keep working. ### 2. A client app registration for Claude Entra has no Dynamic Client Registration, so Claude cannot register itself. Create a separate registration for it: - **Authentication → Add a platform → Web**, redirect URI `https://claude.ai/api/mcp/auth_callback`. - **Certificates & secrets:** create a client secret (it is a confidential client). Note its expiry. The connector stops working when it expires. - **API permissions → Add a permission → APIs my organization uses →** the API registration → **Delegated** → `mcp.access`, then **Grant admin consent** for the tenant. ### 3. Beacon configuration ```csharp builder.Services.AddBeaconJwtAuthentication(x => { x.EnableBearerAuthentication = true; x.Validation.JwksEndpoint = "https://login.microsoftonline.com//discovery/v2.0/keys"; x.Validation.ValidIssuers = ["https://login.microsoftonline.com//v2.0"]; x.Validation.ValidAudiences = ["", "api://"]; }); ``` ```jsonc { "Beacon": { "PublicBaseUrl": "https://", // resource = https:///beacon/mcp "Mcp": { "ProtectedResource": { "AuthorizationServers": ["https://login.microsoftonline.com//v2.0"], // or, instead of listing it: "DeriveAuthorizationServerFromJwtIssuer": true "ScopesSupported": ["api:///mcp.access"] }, "Callers": { "AllowedTenants": [""], "Users": { "Enabled": true, "RequiredRoles": ["Beacon.Mcp.User"], "ProjectIds": [1] } } } } } ``` | Setting (`Beacon:Mcp:ProtectedResource`) | Default | Meaning | |---------|---------|---------| | `AuthorizationServers` | empty | Issuers published as `authorization_servers`, verbatim and in order (clients use the first). Use the **v2** issuer: it equals the `issuer` in Entra's v2 metadata and the `iss` of the tokens. Wins over everything below | | `DeriveAuthorizationServerFromJwtIssuer` | `false` | With no explicit list, publish the bearer JWT issuer (`Validation.ValidIssuer(s)`). With several, the one ending in `/v2.0` is chosen; templated issuers (`{tenantid}`) are skipped | | `ScopesSupported` | empty | Scopes published in `scopes_supported` (and the server card) instead of `Execute` | | `ResourceName` | empty | The `resource_name` clients show (and the server card's `name`, the server name at `initialize`); empty publishes `Beacon MCP` | | `ResourceDocumentation` | empty | The `resource_documentation` URL (and the server card's `documentation`), https; empty publishes Beacon's MCP docs | With none of these set the document lists Beacon's OIDC authority when SSO is on, and the `Execute` scope. An explicit authorization server without `ScopesSupported`, or derivation with no usable issuer, is logged as a warning at startup. The `WWW-Authenticate` challenge, the `resource` in the metadata and the server card's `endpoint` and `resource_metadata` all derive from the same `PublicBaseUrl` (or the forwarded request host when it is unset), so they always agree. ### 4. Add the connector in Claude :::caution[Verify against Anthropic's current connector docs] This repository cannot test Claude's UI, so the steps below are a best reading of it at the time of writing. Check Anthropic's current custom-connector documentation for menu names, the redirect URI, and plan or admin requirements. ::: - In claude.ai, **Settings → Connectors → Add custom connector** (an organization owner may have to add it for the organization first). Enter the server URL, `https:///beacon/mcp`. - Open **Advanced settings** and enter the **OAuth Client ID** and **OAuth Client Secret** of the Claude client registration from step 2. - Connect. Claude opens the Entra sign-in, the user consents (or admin consent already covers it), and the tools appear. Claude Desktop uses the connectors of the signed-in claude.ai account. Things to check when the connection fails, none of which Beacon can see: - **PKCE advertisement.** Entra's metadata does not publish `code_challenge_methods_supported`, although Entra supports PKCE (`S256`). The MCP authorization spec tells clients to refuse an authorization server that does not advertise it; whether Claude enforces that for Entra is for Anthropic's docs to say. - **Refresh tokens.** If Claude asks the user to sign in again every hour, add `offline_access` to `ScopesSupported` (clients request the listed scopes verbatim). - **`AADSTS9010010`**: the `resource` URL Claude sent is not an Application ID URI of the API registration, or a requested scope belongs to another registration. Compare the `resource` in `/.well-known/oauth-protected-resource` with the registration's `identifierUris`. - **`401` with `error="invalid_token"`**: Beacon rejected the token: check `iss`/`aud` against `ValidIssuers` / `ValidAudiences` (a v2 token's `aud` is the client id GUID). A malformed bearer value gets the same `401`, never a `500`. - **`403`**: the token is valid but the user is not an accepted caller (`Users:Enabled`, `RequiredRoles` / `RequiredGroups`, `AllowedTenants`). - **`503` with `Retry-After`**: Beacon could not fetch the Entra JWKS and has no usable keys (none cached, or the last good set is more than 24 hours old). After a failed fetch, the next one is tried 30 seconds later. ## Audit and privacy Every MCP tool call writes an audit row, as before. For a JWT caller, the row also records: - `CallerKind`: `User` or `System`. - `CallerHash`: the lowercase hex HMAC-SHA256 of `{tid}:{subject}`, where the subject is the token's `oid`, else its `azp` / `appid`, else its `sub`. It is stable per identity and never the raw value. The HMAC key is derived for this purpose only from `Beacon:Mcp:CallerHashKey` when it is set, else from `Beacon:EncryptionKey`, so someone who holds the tenant's list of `oid`s cannot reverse the hashes. Without `CallerHashKey`, changing the encryption key changes every caller's hash, so rows written before and after the change no longer group together: before changing the encryption key, set `Beacon:Mcp:CallerHashKey` (a secret) to the key in use now, and the hashes stay the same (see [Configuration → Replacing the Key](https://moberghr.github.io/beacon/getting-started/configuration/#replacing-the-key)). - `UserId`: the provisioned Beacon user, in user mode. Logs carry only the caller hash, never the raw `oid`, email or token. ## Replacing the mapper A host can replace the default mapping. For example, it can map an Entra user to its own admin user and permission set. To do this, register an `IMcpCallerMapper` before `AddBeaconServices`, which registers the default with `TryAdd`, or replace the registration afterwards: ```csharp builder.Services.AddScoped(); ``` The mapper receives the raw token claims (`oid`, `tid`, `groups`, and so on) and returns an `McpCaller`, or `null` to reject the caller. `ConfiguredMcpCallerMapper` is public, so a custom mapper can delegate to it and adjust the result. Inject `McpCallerSubjectHasher`, which `AddBeaconServices` registers as a singleton, to produce the same audit hash. ## Upgrading from 4.5: project access - **User callers need membership on restricted projects.** A user caller reaches a configured project that is restricted only while its Beacon user is a member of it; open configured projects are reached as before. Add the provisioned users as members of the restricted projects they are configured for. - **User callers without a Beacon user reach only their configured open projects** (before: every configured project). - **A disabled or archived user's caller reaches nothing from its next call**, also within the minute its mapping is cached. - **System callers are unchanged**: they reach their configured projects, restricted or not. - **User callers with a Beacon user need Editor to run SQL tools.** `ask`, `query`, `dry_run` and `feedback` need the user's role in the project to be Editor or above (global role, and membership on a restricted project); the catalog tools need Read only. Users provisioned with the default `DefaultRoleName` `Viewer` now read the catalog but are refused the SQL tools: give them the Editor role (and Editor membership on restricted projects) if they should run SQL. - **Host endpoint and saved-query tools** follow the same access: host endpoint tools need the caller to reach the host's project, and saved-query tools are listed only where the caller may run saved SQL. - **New projects are restricted.** A project created after the upgrade by an active Beacon user is restricted, so a user caller configured for it reaches it only once its provisioned Beacon user is a member, with an Editor membership to run SQL tools. Checklist for provisioned users: the default `Viewer` role reads the catalog only; restricted configured projects need membership; the SQL tools need Editor (global role and membership). ## Upgrading from 4.3 The gateway release changes how JWT callers reach MCP. Before upgrading a host that accepts bearer tokens: - **JWT callers on `/beacon/mcp` now need `Beacon:Mcp:Callers`.** A validated token alone no longer reaches MCP: without a matching system or accepted user the caller gets `403`. Configure `Systems` / `Users` for every agent platform that calls MCP with a token. API keys are unaffected. - **Reserved claims are stripped from tokens on every route.** `allowed_projects`, `scope`, `auth_method`, `api_key_id` and `caller_*` inside a token are discarded on `/beacon/api/*` too, so a token can no longer carry its own project restriction or scope. - **New mandatory settings once any caller is configured:** `Beacon:Mcp:Callers:AllowedTenants`, bearer `Validation.ValidIssuer(s)` and `Validation.ValidAudience(s)` with validation on, and either `RequiredRoles` / `RequiredGroups` or `Users:AllowAnyTenantUser = true` for user mode. The host fails at startup with a message that names what is missing. - **App-only tokens need a positive signal**: the `idtyp` optional claim or an app role (see [Token classification](#token-classification)). - **New migrations**, in both providers: `AddMcpAuditCallerIdentity` (audit caller kind/hash), `AddHostManagedDataSources`, `AddProjectImportedDocuments`, `AddQueryMcpTools` and `AddHostManagedProjects` (`Project.HostManagedKey`). Apply them before the new version starts. ### Hosts whose cookie authentication redirects A host whose MVC cookie authentication answers an unauthenticated request with a `302` to its login page breaks MCP clients: an MCP client cannot follow that, and an agent then sees an HTML login page instead of an auth error. Make the cookie events return status codes for the MCP path: ```csharp options.Events.OnRedirectToLogin = ctx => { if (ctx.Request.Path.StartsWithSegments("/beacon/mcp")) { ctx.Response.StatusCode = StatusCodes.Status401Unauthorized; return Task.CompletedTask; } ctx.Response.Redirect(ctx.RedirectUri); return Task.CompletedTask; }; options.Events.OnRedirectToAccessDenied = ctx => { if (ctx.Request.Path.StartsWithSegments("/beacon/mcp")) { ctx.Response.StatusCode = StatusCodes.Status403Forbidden; return Task.CompletedTask; } ctx.Response.Redirect(ctx.RedirectUri); return Task.CompletedTask; }; ``` --- # MCP observability and audit > Every MCP tool call is auditable, correlated to a W3C trace, and optionally exported through OpenTelemetry, without Beacon choosing your telemetry backend. Source: https://moberghr.github.io/beacon/mcp/observability-and-audit/ Every tool call through the [MCP server](https://moberghr.github.io/beacon/mcp/server/) (the built-in tools, saved-query tools and [host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/)) writes one `McpAuditLog` row and emits one structured log event. That includes calls that fail: a tool's own validation errors (an unresolvable `project_id`, a bad `format`, an `offset` out of range), project-access refusals, calls refused for a missing `Execute` scope (audited by the scope filter, before any tool runs) and calls whose project access could not be determined. An unknown `q_` is audited by the saved-query tools. A tool name that belongs to no tool family is answered as a JSON-RPC error (`Unknown tool`) and writes no row. With `Beacon:Mcp:Audit:Required = true`, a result whose row could not be written, or a successful result that produced no row, is withheld rather than returned; see [Fail-closed audit](#fail-closed-audit). Beacon never picks an exporter or a backend itself: it tags the MCP SDK's own span and records its own metrics, and a host wires those into whatever it already uses (OTLP to an ADOT collector, Phoenix, or another backend). ## Configuration ```jsonc { "Beacon": { "Mcp": { "Audit": { "Required": false, // fail closed: withhold a result whose audit row could not be written "RetentionDays": null, // null or 0 = keep forever (default); 365 recommended; > 0 = purge older rows daily "RequestIdHeader": "X-Request-Id", // upstream correlation header; null/empty = read none "RequireContent": false // pin content retention on, so audit content cannot be switched off at runtime } }, "Telemetry": { "CaptureContent": false // put the tool input on the current span: double opt-in, see below } } } ``` | Setting | Default | Meaning | |---|---|---| | `Beacon:Mcp:Audit:Required` | `false` | When `true`, a tool result whose audit row failed to write is replaced with an error instead of being returned. While it is `false` the host logs a Warning at startup: such a call is then recorded only by the `Beacon.Audit` 9100 event (`Persisted=false`) and the audit write failure metric. | | `Beacon:Mcp:Audit:RetentionDays` | `null` | Days audit rows are kept. `null` or `0` keeps them forever (`365` recommended); a negative value fails options validation at startup. Under `Beacon:Retention:Locked` it must be greater than 0. | | `Beacon:Mcp:Audit:RequestIdHeader` | `X-Request-Id` | The upstream header read into `UpstreamRequestId`. Must be a valid HTTP header token, or startup fails. Set to `null`/empty to read none. | | `Beacon:Mcp:Audit:RequireContent` | `false` | When `true`, pins `RetainQueryContent` on globally and for every project, so nobody, an Admin included, can switch the audit's question and SQL text off at runtime. See [Keeping audit content](#keeping-audit-content-the-requirecontent-lock). | | `Beacon:Telemetry:CaptureContent` | `false` | Puts the tool input on the current span as `beacon.tool.input`, only where the project's own content retention also allows content (double opt-in, never from this setting alone). | ## Knowing the retention jobs run Beacon schedules no maintenance job itself: the audit purges and the security audit verification run only where the host schedules them (the sample host does, with Warp recurring jobs `mcp-audit-retention`, `security-audit-retention` and `security-audit-verify`). Each of those services records a heartbeat (`MaintenanceHeartbeats`, one row per job: its name and the time it last ran to its end) when it finishes. At startup the host looks the heartbeats up, bounded and best effort, and logs a Warning for each job that should be running and has no recorded run, or none within 36 hours (6 for the verification): a purge counts only while its `RetentionDays` is set. A first start has no heartbeat yet and warns once. `IRetentionScheduleCheck` runs the same check for a host that wants it on a schedule of its own. ## What's on an audit row `McpAuditLog` (one row per tool call) carries, beyond the tool name, timing, result row count and (redacted, per the project's retention decision) parameters/error: | Column | Source | |---|---| | `CorrelationId` | Minted by Beacon for every audited call (32 hex characters) and never read from a request: the one identifier on the row, and on the 9100 event, that a caller cannot choose. `null` on rows written before it existed. | | `TraceId` / `SpanId` | `Activity.Current` (the MCP SDK's `tools/call` span) when one is current; otherwise `null`. | | `McpSessionId` | The `Mcp-Session-Id` transport header. `null` for stateless clients. | | `UpstreamRequestId` | Supplied by the caller and not verified: use `CorrelationId` as the identifier Beacon vouches for. The header named by `Beacon:Mcp:Audit:RequestIdHeader`, accepted only when it is at most 128 characters of `[A-Za-z0-9._:-]`; anything else is dropped, which blocks log injection. | | `ApiKeyId` | The `api_key_id` claim of an API-key caller. No foreign key: the audit must outlive a deleted key. | | `CallerKind` / `CallerHash` | Set for a mapped [Entra ID caller](https://moberghr.github.io/beacon/mcp/entra-callers/); `null` for API keys and cookie sessions. | | `ContentHash` | A keyed hash of the call's parameters as received, written **whether or not** the parameters themselves are kept, so calls with the same input can be matched (and a disclosed input checked against the trail) without keeping it. See [Keyed hashes](#keyed-hashes). | | `DataSourceId`, `ResultRowCount` | For `ask`: the data source and final row count of a single-source answer; `null` for a cross-source answer, whose sources and row counts are on its statements. | ### The statements a call ran Each audit row also has one `McpAuditStatement` child row per statement the call sent to a data source, in the order they ran, on the success and the failure path alike, whatever the learning settings. The executors that reach a data source record them (the query execution service, the cross-source service and the value-grounding probes), and the pipeline that runs them names why: | Purpose | Statement | |---|---| | `initial` | The first execution of the SQL an `ask` generated. | | `repair` | An execution of a repaired statement, after an execution error or an empty result. | | `candidate` | A self-consistency candidate executed to vote on its result. | | `probe` | A value-grounding probe for a literal of the question. | | `dry_run` | A validation on the data source without running the statement (`EXPLAIN` and the like), by `dry_run` or before a cross-source step. | | `source_step` | One data source's statement of a cross-source answer (the in-memory join that combines them reaches no data source and is not a statement). | | `query` | A statement the `query` tool ran as asked, or one run outside any named purpose. | A statement row holds its data source, its purpose, `SqlHash` (a keyed hash of the statement text, **always** kept), `Sql` (the text: **content**, kept only while the call's project retains query content; the content-retention interceptor strips it otherwise), the tables it names, its row count, its duration and the class of the error it ended with (never the message; an empty error message counts as no error). Statement rows are deleted with their audit row, so the retention purge covers them. A row never reads as the whole of what a call ran when it is not: - **At most 100 statements a call.** Further ones are counted, not stored: the call's statements then end with a marker row of purpose `truncated` (no data source and no text) whose `RowCount` is the number of statements not stored, and a Warning names the tool and the count. - **Table lists.** A statement's table list holds as many names as fit its column; when not all fit, its last entry is `[+n more]`. A statement whose tables could not be read from its text has the list `["[unparsed]"]`: unknown, not empty. The tables a call's structural parameters name (kept when its content is not) leave these markers out. ### Keyed hashes `ContentHash` and `SqlHash` are lowercase hex HMAC-SHA256 values under a key derived, for this purpose only (HMAC of a fixed label), from the secret the MCP caller hashes come from: `Beacon:Mcp:CallerHashKey`, else `Beacon:EncryptionKey`. An absent input has a hash of its own, apart from the empty text's. The secret itself is never the key, and nobody without it can test guesses against the stored hashes. The hashes stay comparable across an encryption-key change while `Beacon:Mcp:CallerHashKey` is set (see [Replacing the Key](https://moberghr.github.io/beacon/getting-started/configuration/#replacing-the-key)). ### Sealed rows Every row is sealed as it is saved: an HMAC-SHA256 over its fields and the statements it holds, under the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/#the-mcp-audit-rows)'s integrity keys (`Hmac`, `HmacKeyId`, `SealVersion`, `AuditGuid`). Beacon refuses to change or delete a row or a statement, and the scheduled security audit verification checks the rows written since its last run, checks again a rolling slice of the rows it checked before, and counts the rows up to its checkpoint, reporting changed and deleted ones at Error (9111) and writing a checkpoint (9112). Rows written before seals were introduced are reported as unsealed. ## The `Beacon.Audit` log event catalogue Every event below is logged at category **`Beacon.Audit`**, structured, with identifiers and counts only, never parameters, SQL, a question, or a raw error message (only its class, via `McpContentRedactor.ErrorClassOf`). Point your log sink (CloudWatch, a SIEM) at this category to get the audit trail without reading Beacon's database. | EventId | Name | Level | When | Notable fields | |---|---|---|---|---| | 9100 | `McpToolAudited` | Information | Once per `McpAuditService.LogToolCallAsync` call, including when the row failed to save. | `Tool`, `AuditId` (null if not persisted), `Persisted`, `ProjectId`, `DataSourceId`, `UserId`, `ApiKeyId`, `CallerKind`, `CallerHash`, `DurationMs`, `Rows`, `ErrorClass`, `CorrelationId`, `TraceId`, `McpSessionId`, `UpstreamRequestId` | | 9101 | `McpToolResultWithheld` | Warning | `Beacon:Mcp:Audit:Required = true` and the audit write for that call failed, or a successful result was not audited (MCP transport and Playground/REST tool runs). | `Tool`, `TraceId` | | 9102 | `McpAuditPurged` | Information | The retention purge runs with `RetentionDays` set. | Deleted count, cutoff | | 9103 | `McpAuditRead` | Information | `GET /beacon/api/mcp/audit` is called. | `RequestedByUserId`, `From`, `To`, `ProjectId`, `HasToolFilter` (bool: the admin-supplied `Tool` filter is free text, so only its presence is logged, never the value), `UserId`, `HasCallerHashFilter` (bool, same reasoning), `Page`, `PageSize`, `ReturnedCount` | The same category carries the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/#shipping-to-a-siem)'s events 9104 to 9114: a copy of every committed security audit event, reads and verifications of the trail, purges, write and verification failures, the checkpoints of its scheduled verification, and the failures (9111) and checkpoints (9112) of the MCP audit rows' verification. ### Row data sent to the LLM provider Prompts that carry data read from a data source are logged at category `Beacon.AI.Services.LlmProviders.LlmRowDataEgress`, event **7123** `RowDataSentToLlm`, identifiers only, never a value or prompt text, and no table is written: | Level | When | Message fields | |---|---|---| | Information | A request carrying row data is handed to the LLM provider (written before the provider answers, since the data may leave even when it then fails). | `Categories`, `DataSourceIds`, `ProjectId`, `CallerId` | | Warning | The provider then fails that request. | the same, plus `Failure`: the exception's type and error code, never its message | | Information | A `get_query_context` response returns sample values or value matches to the MCP client. | `Categories`, `DataSourceIds`, `ProjectId`, `CallerId` | The categories are `sample-values`, `value-matches`, `column-values` (documentation enum discovery), `result-rows` (the eval judge), `database-errors` (lesson extraction) and `stored-documentation` (business-domain, data-model and glossary sections, which can list discovered values). `CallerId` is the caller's Beacon user id; background work names itself: `learning-aggregation`, `documentation-indexing`, `pattern-replay`, an eval run's starting user or `eval-run`, documentation generation's requesting user; and a caller that cannot be resolved is `unknown` (the lookup failure is logged as a Warning by type, and never fails the request). See [What reaches the LLM provider](https://moberghr.github.io/beacon/mcp/server/#what-reaches-the-llm-provider). ## Span attributes and metrics The MCP SDK already opens one span per `tools/call` (activity source `Experimental.ModelContextProtocol`, tagged `gen_ai.tool.name`, `mcp.method.name`, `mcp.session.id`, `error.type`, `jsonrpc.request.id`) and records `mcp.server.operation.duration`. Beacon does not open a duplicate span; it tags the SDK's own current activity. The SDK's spans are exported only when the host adds them with `AddBeaconMcpSdkTracing()` (see [Turning tracing on](#turning-tracing-on)): | Tag | Meaning | |---|---| | `beacon.project.id`, `beacon.data_source.id` | The resolved project/data source. | | `beacon.caller.kind`, `beacon.caller.hash` | The mapped caller (Entra callers only). | | `beacon.result.rows` | Rows returned, when the tool reports a count. | | `beacon.error.class` | The redacted error class (never the raw message). | | `beacon.audit.persisted` | Whether the audit row was written. | | `beacon.tool.input` | The tool input, truncated to 4000 characters. **Double opt-in only**. see below. | On the `Beacon` meter: | Instrument | Kind | Tags | |---|---|---| | `beacon.mcp.tool.calls` | Counter | `gen_ai.tool.name`, `beacon.outcome` (`success`\|`error`), `error.type`, `beacon.caller.kind` | | `beacon.mcp.tool.rows` | Histogram | `gen_ai.tool.name` | | `beacon.mcp.audit.write_failures` | Counter | `gen_ai.tool.name` | | `beacon.error_redaction.fallbacks` | Counter | none: a database error that could not be [screened](https://moberghr.github.io/beacon/mcp/server/#database-errors) in time and was replaced as a whole | Duration is not duplicated on a Beacon instrument; the SDK's own `mcp.server.operation.duration` already covers it. Tool names can be caller-chosen (an unknown `q_` saved-query name, for instance). The 9100/9101 `Tool` log field carries the tool name only when it matches `^[A-Za-z0-9_.:\-]{1,200}$`; anything else is reported as the constant ``. This blocks log injection; logs are not a cardinality problem, so a charset/length bound is enough there. The `gen_ai.tool.name` **metric** tag is drawn from a closed set instead: a charset bound alone doesn't fix cardinality, because a well-formed but caller-chosen name (`q_1`, `q_2`, ... each a distinct saved-query name) would still open a new time series per name. A built-in tool (the attribute tools in `McpToolCatalog`, plus the fixed `search_saved_queries` / `run_saved_query` / `search_api` / `call_api` dispatcher names) keeps its own value; an unknown name starting with `q_` or `api_` (including a write tool's `api_:write` audit name) collapses to the fixed bucket `q_*` / `api_*`; anything else is ``. That is a handful of possible values regardless of how many distinct tool names callers invent. The audit row itself keeps the tool name as before, unredacted. > The MCP SDK's own meter (also exported by `AddBeaconInstrumentation`) tags its instruments (including > `mcp.server.operation.duration`) with `gen_ai.tool.name` set to the **raw** caller-chosen name; Beacon does not > control that tag. A host worried about cardinality on the SDK's own meter can drop or re-bucket that tag with an > OpenTelemetry [View](https://opentelemetry.io/docs/specs/otel/metrics/sdk/#view) in its `MeterProviderBuilder`. ### LLM completions and ask stages Beacon opens its own spans on the `Beacon` activity source for the work behind a tool call. **No tag carries content**: never the prompt, the system prompt, the completion, SQL or the question, whatever `Beacon:Telemetry:CaptureContent` says. Tags hold names, counts and flags only. `beacon.llm.completion` (kind Client): one span per LLM completion, opened around the request queue, so it covers the wait for a slot, the retries and the provider call. | Tag | Meaning | |---|---| | `gen_ai.provider.name` | The configured provider kind (`openai`, `claude`, `azureopenai`, `bedrock`), read from the same configuration snapshot as the provider that serves the call. | | `gen_ai.request.model` | The configured model (a configuration value), from that same snapshot. | | `gen_ai.response.model` | The model the provider reports, only when it reads as a model name (at most 128 letters, digits and `. - _ : / @`); absent otherwise. | | `beacon.llm.operation` | What the completion is for, a fixed name such as `sql.generation`, `sql.self_consistency`, `sql.repair`, `ask.intent`, `ask.routing`, `ask.knowledge_answer`, `doc.indexing`, `learning.lesson`, `documentation.generate`, `actor.think`. Absent when the caller set none. | | `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens` | Tokens the provider reports. | | `gen_ai.usage.cache_read.input_tokens`, `gen_ai.usage.cache_creation.input_tokens` | Prompt-cache reads and writes, where the provider reports them (Claude, also on Bedrock); otherwise 0. | | `beacon.llm.cache_hint` | Whether the request marked any message as a prompt-cache breakpoint. Only the Claude provider acts on it; Bedrock does not request prompt caching yet, and OpenAI and Azure OpenAI cache on their own. | | `beacon.llm.truncated` | Whether the answer stopped at the token limit. | | `beacon.outcome` | `success`, `error`, or `cancelled` when the call was cancelled (a cancellation sets no error status and no error type). | | `error.type`, `beacon.error.root_type` | On an error only: the exception's type name and the innermost exception's type name (a provider wraps its failures, so the root shows the cause). Never a message. The span's status is Error. | `beacon.ask.`: one span per stage of an `ask`, tagged `beacon.ask.stage` and counts only. A completion made in a stage is its child span. A stage that throws is marked like a completion: `beacon.outcome` `cancelled` on a cancellation, otherwise `error` with error status, `error.type` and `beacon.error.root_type` (a failing self-consistency vote is marked so although the `ask` goes on with the single candidate). | Span | Extra tags | |---|---| | `beacon.ask.intent` | `beacon.ask.intent` (`DataQuery` or `Knowledge`) | | `beacon.ask.routing` | `beacon.ask.sources` (data sources considered) | | `beacon.ask.generation` | `beacon.ask.tables` (distinct tables the candidate reads), `beacon.ask.context_chars` (characters of the schema context sent), `beacon.ask.tables_shown` and `beacon.ask.tables_omitted` (tables the context describes, and tables `Ceilings:AskSchemaContextChars` left out; these three are set before the call, so a failed generation carries them too) | | `beacon.ask.self_consistency` | `beacon.ask.candidates` (candidates voted on) | | `beacon.ask.repair` | `beacon.ask.repair.trigger` (`schema`, `lint`, `dry-run`, `execution`, `empty-result`), `beacon.ask.produced_sql` (whether a statement came back) | | `beacon.ask.execution` | `beacon.ask.statement.purpose` (`initial`, `repair`, `candidate`), `beacon.result.rows`, `beacon.outcome` | On the `Beacon` meter: | Instrument | Kind | Tags | |---|---|---| | `beacon.llm.calls` | Counter | `gen_ai.provider.name`, `gen_ai.request.model`, `beacon.llm.operation`, `beacon.outcome` (`success`, `error`, `cancelled`), `error.type` (errors only) | | `beacon.llm.tokens` | Histogram | `gen_ai.provider.name`, `gen_ai.request.model`, `beacon.llm.operation`, `gen_ai.token.type` (`input`, `output`, `cache_read`, `cache_write`); a type with no tokens records nothing | A completion made outside a request queue's provider (the admin connection test) has no span. ### Content on spans is a double opt-in `beacon.tool.input` is added to the current span only when **both**: 1. `Beacon:Telemetry:CaptureContent = true`, and 2. the calling project's effective settings retain query content (`ContentRetentionDecision.RetainQueryContent`), which a deployment-level `Beacon:Mcp:ForceNoContentRetention` lock always pins off. Log events and metrics never carry content, regardless of either setting. The MCP SDK writes an error result's content, or an exception's message, into its span's status description. `AddBeaconMcpSdkTracing()` clears every such description when the span ends, before the provider's processors and exporters see it; the status code (`Error`) and `error.type` stay. Keep the SDK's `ModelContextProtocol` log category above `Trace` in production: at `Trace` the SDK logs whole requests and responses. ## Turning tracing on Beacon.Api exposes `BeaconTelemetryBuilderExtensions.AddBeaconInstrumentation()` for both `TracerProviderBuilder` and `MeterProviderBuilder`. It only depends on `OpenTelemetry.Api`; it adds the `"Beacon"` source to the tracer provider and the `"Beacon"` meter plus the MCP SDK's own `Experimental.ModelContextProtocol` meter to the meter provider. `AddBeaconMcpSdkTracing()` adds the SDK's `tools/call` spans (the spans Beacon tags) with their status descriptions cleared; without it those spans, and Beacon's tags on them, are not exported. **Beacon never picks an exporter.** The sample host (`Beacon.SampleProject/Program.cs`) enables OpenTelemetry only when `OTEL_EXPORTER_OTLP_ENDPOINT` is set: ```csharp if (!string.IsNullOrWhiteSpace(builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"])) { builder.Services.AddOpenTelemetry() .ConfigureResource(x => x.AddService(serviceName: builder.Configuration["OTEL_SERVICE_NAME"] ?? "beacon")) .WithTracing(x => x .AddAspNetCoreInstrumentation(y => y.Filter = z => !z.Request.Path.StartsWithSegments("/beacon/api/health")) .AddHttpClientInstrumentation() .AddBeaconInstrumentation() .AddBeaconMcpSdkTracing() .AddOtlpExporter()) .WithMetrics(x => x .AddAspNetCoreInstrumentation() .AddHttpClientInstrumentation() .AddBeaconInstrumentation() .AddOtlpExporter()); } ``` With no `OTEL_EXPORTER_OTLP_ENDPOINT`, none of this runs: there is no OpenTelemetry registration at all. Everything else is driven by the standard `OTEL_*` environment variables Beacon never reads itself: `OTEL_SERVICE_NAME`, `OTEL_TRACES_SAMPLER`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, and so on. No endpoint, header or credential is ever hardcoded in Beacon. A host that embeds Beacon (rather than running the sample host) calls `.AddBeaconInstrumentation()` (and, for the SDK's spans, `.AddBeaconMcpSdkTracing()`) from its own tracing/metrics configuration the same way, after adding whichever OpenTelemetry SDK and exporter packages it needs. ### Trace continuity into host endpoint dispatch `HostEndpointDispatcher` runs the dispatched host endpoint under `ExecutionContext.SuppressFlow()` (inside a fresh `Task.Run`), so the MCP request's ambient execution context, including `Activity.Current`, does not flow into the synthetic request. Inside the dispatched host code, `Activity.Current` is **`null`**: host `ILogger` scopes built there carry no `TraceId`/`SpanId`, and an outbound `HttpClient` call the host code makes starts a brand-new trace of its own rather than a child span of the MCP call. The only link between the two traces is `HttpContext.TraceIdentifier`. Before suppressing flow, the dispatcher reads `Activity.Current?.Id` on the still-flowing caller side and carries it in explicitly as the synthetic request's `TraceIdentifier` (falling back to a synthetic `beacon-mcp-…` id when no activity was current). `Activity.Id` is W3C traceparent-format, `00-<32 hex trace id>-<16 hex span id>-01`, so the 32-hex segment in the middle of `HttpContext.TraceIdentifier` is exactly the audit row's `TraceId`. Host code that wants to correlate its own logs or spans to the MCP call has to read `HttpContext.TraceIdentifier` and parse out that segment; it cannot rely on `Activity.Current` being set. ## CloudWatch and collectors **JSON console logs, no OTel needed.** Set `Logging:Console:FormatterName=json` and `Logging:Console:FormatterOptions:IncludeScopes=true`. ASP.NET Core's `ActivityTrackingOptions` already defaults to including `TraceId`/`SpanId`/`ParentId`, so every log line (including `Beacon.Audit` events) carries them once scopes are on. Ship that JSON stream with `awslogs` (or any agent that reads container stdout) straight into CloudWatch Logs; this path needs no OTel exporter at all. **An ADOT collector**, receiving OTLP from Beacon and exporting to X-Ray/CloudWatch (or to Phoenix for local/dev tracing), looks like: ```yaml receivers: otlp: protocols: http: grpc: processors: batch: exporters: awsxray: # or: otlp/phoenix: { endpoint: "http://phoenix:6006/v1/traces" } awsemf: # CloudWatch metrics service: pipelines: traces: receivers: [otlp] processors: [batch] exporters: [awsxray] metrics: receivers: [otlp] processors: [batch] exporters: [awsemf] ``` Point `OTEL_EXPORTER_OTLP_ENDPOINT` at the collector; the collector holds the AWS credentials or the Phoenix endpoint, never Beacon. ## Fail-closed audit With `Beacon:Mcp:Audit:Required = true`, `McpAuditCallToolFilter` wraps every tool call (attribute tools, saved-query tools and host-endpoint tools alike). The tool's result is replaced with an error, *"Result withheld: the audit record for this call could not be written. Contact your Beacon administrator."*, and a `Beacon.Audit` 9101 event is logged when either: - the audit row for that call failed to write, or - the tool returned a **successful** result and no audit row was written for it at all. An error result that was not audited passes through unchanged; it carries no data. With `Required = false` (the default), results are always returned unchanged regardless of the audit outcome. This is enforced **after execution**: the underlying query has already run by the time the filter withholds the result, and the audit and signal writes are never skipped. The guarantee is *no unaudited data leaves Beacon*, not *no unaudited execution*. The audit row itself is written synchronously, before the filter answers, never through a queue: a tool keeps its call on the request's `McpToolCallRecorder` (tool, caller, input, project, data source, rows or error, tables, and for `ask`, `query` and `dry_run` only, the learning signal), and the filter writes the row and then the signal after the tool returned or threw, never while it runs, so the row holds the call's final outcome. A tool that throws after starting its record is recorded as failed and answers the same caller-safe error the tools' own error handling gives (a database error's text through the database error redactor while PII detection is on); a protocol error is written first and then answered as the protocol error it is. A tool that wrote its own row is never given a second one (its signal still is), and one whose own write failed is written again. A tool that throws before any row was written, including a call to an unknown tool, still gets one (tool, caller and the error, the error's text only when the call's project is known and its content retention keeps it), its exception unchanged. The Playground and the "run tool" REST path (`POST /beacon/api/mcp/tools/run`, `RunMcpToolHandler` → `McpPlaygroundService`) call tools directly, not through the SDK's request pipeline, so they run them through the same audit path (`McpAuditCallToolFilter.InvokeAsync`) against their own request scope: the same rows, and with `Required = true` the same withheld error and 9101 event under the same conditions. A playground call refused before any tool ran is recorded through the same recorder. ## Keeping audit content: the RequireContent lock Whether the audit keeps question and SQL text follows each project's content retention (`RetainQueryContent`, see [Content retention](https://moberghr.github.io/beacon/mcp/server/#content-retention)), which an Admin or SecurityAdmin can switch off on the MCP Settings page. A deployment that must keep that evidence sets `Beacon:Mcp:Audit:RequireContent: true`: it pins `RetainQueryContent` on, globally and for every project, like the other [deployment locks](https://moberghr.github.io/beacon/getting-started/configuration/#deployment-locks). - A save that would switch content retention off, globally or as a project override, is refused with `409` (`/errors/setting-locked`, naming `Audit:RequireContent`), an Admin's too. The MCP Settings page shows the switch disabled with the reason, and `GET /beacon/api/mcp/settings/locks` lists the lock in `locks` (`{ "fieldName": "RetainQueryContent", "lockName": "Audit:RequireContent" }`). - A stored value it overrides (retention switched off before the lock) is kept, has no effect, and is named in the startup warning about overridden settings. - `RequireContent` and `Beacon:Mcp:ForceNoContentRetention` pin the same setting in opposite directions: with both on, the host fails to start. With the lock on, every audited call keeps its question and the SQL of every statement it ran until the audit retention purges them. Plan the privacy review for that, and set `Beacon:Mcp:Audit:RetentionDays` accordingly. When a project's retention setting cannot be read while the lock is on, the call keeps its content (logged at Error); without the lock it is audited structurally, as before. The audit rows and their statements are [sealed](#sealed-rows): an edit made at the database is found by the next scheduled verification, as in the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/#what-the-trail-does-and-does-not-prove); a deleted row is found by the row count once a run has checked it. Ship the `Beacon.Audit` events to a store you control for a copy the database cannot change. ## Retention Audit rows are kept forever by default. Set `Beacon:Mcp:Audit:RetentionDays` to a positive number (`365` is recommended) to purge rows older than that window. The purge itself is `IMcpAuditRetentionService.PurgeExpiredAsync` (`src/Beacon.Core/Services/Retention/McpAuditRetentionService.cs`), a no-op while `RetentionDays` is `null` or `0`, otherwise one `ExecuteDeleteAsync` plus a `Beacon.Audit` 9102 event with the deleted count and cutoff. A row's statements are deleted with it. The sample host schedules it as the Warp recurring job `mcp-audit-retention` (`30 3 * * *`, `PurgeExpiredMcpAuditLogsJob` / `PurgeExpiredMcpAuditLogsJobHandler` in `Beacon.SampleProject/Warp/Jobs/McpMaintenanceJobs.cs`). A host embedding Beacon schedules this itself, for example with Warp: ```csharp await recurringJobPublisher.AddOrUpdateRecurringJob(new PurgeExpiredMcpAuditLogsJob(), "mcp-audit-retention", "30 3 * * *"); ``` or by calling `IMcpAuditRetentionService.PurgeExpiredAsync(cancellationToken)` from its own scheduler. ## Reading the audit `GET /beacon/api/mcp/audit` is a paged JSON export for an Auditor or an Admin (the [Auditor permission](https://moberghr.github.io/beacon/security/authorization/#functional-permissions)). Rows come back exactly as stored; any retention redaction was already applied at write time, so this endpoint never re-reads or reconstructs content. The rows are not filtered by project membership: an Auditor reads every project's tool calls, including the SQL and question text the retention settings keep, though the Auditor role grants no other project data. Reading the audit is itself audited: each read is recorded first in the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/) (`audit.mcp_audit_read`, with the filters' shape and the page; an identical read by the same actor within 60 seconds once), and fails when that record cannot be written; 9103 (above) follows on the log stream. | Query parameter | Required | Notes | |---|---|---| | `from`, `to` | Yes | UTC. A value without an offset is taken as UTC; a value with an offset is converted to UTC. The range is validated after that normalisation: `to` must not be earlier than `from`, and the span must not exceed 93 days. | | `projectId`, `tool`, `callerHash`, `userId` | No | Exact-match filters. | | `page` | No (default `1`) | Must be `≥ 1`. | | `pageSize` | No (default `100`) | Must be between `1` and `500`. | Each row carries its `contentHash` and its `statements` (`id`, `purpose`, `dataSourceId`, `rowCount`, `durationMs`, `errorClass`, `sqlHash`, and `sql`, which is `null` unless the call's project retained query content), in the order they ran. Any other caller gets `403`. An invalid range or paging parameter gets `400`. The **MCP Audit** page (`/beacon/mcp-audit`, shown to Auditors and Admins) reads it, and the generated TypeScript client exposes it as `getMcpAuditLogs`. Security-relevant actions other than tool calls (sign-ins, users, keys, settings and the rest) are in the separate [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/). --- # MCP gateway for your application > Give AI clients (Claude, Claude Desktop, Cowork, Claude Code) governed access to your own ASP.NET Core application: your users sign in with Entra, your endpoints become tools that run as them, your docs become grounding. One package, about thirty lines, no data sources. Source: https://moberghr.github.io/beacon/mcp/gateway/ This page is the whole recipe for one use case: **an MCP server for your own application.** People connect Claude to `https://your-app/beacon/mcp`, sign in with Microsoft Entra, and Claude calls a set of your existing endpoints **as that person**, through your own authorization, with every call audited. Beacon's data product (data sources, SQL, saved queries, subscriptions) is not part of it. If that is what you want, follow [Integrate Beacon](https://moberghr.github.io/beacon/integrate/) instead. :::tip[Using Claude Code or another coding agent?] With Claude Code, install Beacon's plugin: `/plugin marketplace add moberghr/beacon`, then `/plugin install beacon@moberg-beacon`. Its skills (`beacon-gateway`, `beacon-tool`, `beacon-entra`) follow this guide. With any agent, point it at this guide in one file: ```text wrap Add an MCP gateway to this ASP.NET Core app with Beacon. Read https://moberghr.github.io/beacon/llms-gateway.txt first and follow it step by step. Ask me before every decision it marks as one. ``` ::: ## Decisions to make first An agent should ask these, not guess. | Decision | Options | Default | |---|---|---| | Who may connect | Every active user of your application, or a subset | Every active user your users table knows | | What a tool call may do | Self-service only (the person's own records), or what the person may do in the browser | **Self-service only**: an agent reads other people's free text and sends what it reads to a model | | Which endpoints become tools | Any endpoint with authorization metadata | Reads of the person's own data, plus writes the person would do themselves | | Admin UI for the audit and settings | On (behind one of your policies) or off | On | | Beacon's database | PostgreSQL (with pgvector available) or SQL Server; the app's database in schema `beacon`, or a separate one | The app's own database, schema `beacon` | ## 1. Packages and secrets ```bash dotnet add package Moberg.Beacon.Gateway dotnet add package Moberg.Beacon.Core.PostgreSql # or Moberg.Beacon.Core.SqlServer dotnet add package Moberg.Beacon.Testing # in the test project ``` `Beacon:EncryptionKey` is required: base64 of 32 random bytes (`openssl rand -base64 32`), from user secrets or your secret store, never a committed file. It keys the caller hashes in the audit. Set `Beacon:PublicBaseUrl` to the URL clients use (`https://your-app`): the discovery documents and challenges advertise `/beacon/mcp`. ## 2. Who a token is: `IMcpHostUserResolver` Beacon validates the Entra token (tenant, audience, delegated, scope, client). Your application decides who the person is and what a tool call carries: ```csharp internal sealed class AppMcpUsers(AppDbContext db) : IMcpHostUserResolver { public async Task FindAsync(McpTokenIdentity token, CancellationToken ct) { var users = await db.Users .Where(x => x.ObjectId == token.ObjectId) .Where(x => x.IsActive) .Select(x => new { x.Id, x.DisplayName }) .Take(2) .ToListAsync(ct); // Exactly one, or nobody: an object id two active users carry does not say which of them this is. if (users.Count != 1) { return null; } return new McpHostUser( users[0].Id.ToString(), users[0].DisplayName, [new Claim("app_user_id", users[0].Id.ToString())]); // the claims your own sign-in adds } } ``` - Look users up by `ObjectId` (`oid`), never by `sub`: the token was issued to another client, so its `sub` differs. - Never create or link a user here. Someone who has never signed in to your application is refused. - It runs on every tool call, so deactivating a person in your application cuts MCP off at their next call. ## 3. Program.cs ```csharp // Register any IClaimsTransformation of your own BEFORE the gateway (it runs yours first). builder.Services.AddBeaconMcpGateway(builder.Configuration, x => { x.ProjectName = "app"; // Beacon's project for your tools and docs x.Database(db => db.UsePostgreSql(sp => sp.GetRequiredService())); // or db.UsePostgreSql() with ConnectionStrings:BeaconContext x.Entra(builder.Configuration.GetSection("AzureAd")) // TenantId, ClientId (Microsoft.Identity.Web's section) .AllowClients(builder.Configuration.GetSection("McpAccess:AllowedClientIds")); // optional: only Claude's client x.Users(); x.Tools(y => { y.AllowWriteTools = true; // only if you mark write tools y.BindFromCaller("userId", "app_user_id"); // the person's own id, never the agent's }); x.Docs(y => y.FromEmbeddedResources(typeof(Program).Assembly, "app-docs/")); // optional grounding x.AdminUi(y => // optional { y.HostPolicy = "beacon.admin"; // one of YOUR authorization policies y.HostUserId = z => z.FindFirst("app_user_id")?.Value; y.SignInPath = "/Account/SignIn"; }); }); var app = builder.Build(); app.UseBeaconMcpGateway(); // before UseAuthentication // ... UseStaticFiles (the admin UI's assets), UseAuthentication, UseAuthorization ... app.MapBeaconMcpGateway(); // /beacon/mcp, discovery documents, the admin UI ``` `UseBeaconMcpGateway()` also adds Beacon's security headers and its Host and Origin check on `/beacon` (an OpenID Connect callback always passes the check), as `UseBeaconAuthentication()` does for a full host. See [Security headers and the origin check](https://moberghr.github.io/beacon/integrate/host-setup/#security-headers-and-the-origin-check). What you get, and what you do not have to write: - **Callers:** delegated tokens of your tenant and registration with the `mcp.access` scope (`RequireScope` changes it), whose person your resolver knows. App-only tokens, ID tokens, other tenants and other scopes are refused. Entra roles and groups are never read. - **Tools:** your marked endpoints as `api_`, plus `get_context`, `search` and `get_documentation` for your docs. No SQL tools, no saved-query tools, no data sources. Each tool's description ends with a note that it runs with the caller's permissions in your application; when your endpoints run with less (self-service only, say), word it yourself with `y.DescriptionNote` in `x.Tools(...)` (see [Host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/)). - **Locks:** read-only and no content retention are pinned in code (`RetainToolContent = true` keeps call content). - **Your session never counts on `/beacon/mcp`,** and a bearer token never counts on your routes. The MCP route answers 401/403 itself, never your sign-in redirect: you edit no cookie events. - **Discovery:** `/.well-known/oauth-protected-resource` publishes your tenant's v2 issuer and `api:///mcp.access` plus `offline_access` (so clients refresh instead of re-prompting hourly). It names the server `Beacon MCP` (the server card and `initialize`: `Beacon`) and links Beacon's docs unless you set `Beacon:Mcp:ProtectedResource:ResourceName` and `ResourceDocumentation` (an https URL) to your application's. The server card describes the gateway (your endpoints and documents); `x.ServerDescription` words it yourself. - **The host project** is synced the first time it is needed; the `beacon-mcp-gateway` health check reports Degraded until then. No hosted service, no startup step. - **Admin UI** at `/beacon`: the MCP audit, the MCP settings, the learning views and your project, for holders of `HostPolicy`, signed in with your own sign-in. Every other Beacon endpoint answers 404. ## 4. Migrations Beacon's tables live in schema `beacon`. Apply them from your migration step, not from every web replica: ```csharp await Beacon.Core.PostgreSql.BeaconPostgreSqlMigrator.MigrateAsync(dataSource); // or (connectionString) await Beacon.Core.SqlServer.BeaconSqlServerMigrator.MigrateAsync(connectionString); ``` On PostgreSQL one migration creates the `vector` extension: allow it (Azure Database for PostgreSQL: add `VECTOR` to `azure.extensions`, which replaces the whole list). Sharing your `NpgsqlDataSource` needs connection reset on and multiplexing off; Beacon refuses one that does either. ## 5. Mark endpoints as tools On a minimal API endpoint (`using Beacon.MCP.HostEndpoints;` for `WithBeaconTool`), a controller action, or a handler class whose framework copies class attributes into the endpoint (a Moberg Warp handler; `using Beacon.Core.HostEndpoints;` for `[BeaconTool]`): ```csharp app.MapGet("/api/my/orders", (int userId, OrderService orders) => orders.Of(userId)) .RequireAuthorization() .WithBeaconTool("my_orders", "The signed-in user's orders, newest first: id, placedAt (UTC), status, total.", readOnly: true); [Authorize] [BeaconTool("request_leave", Description = "...", ReadOnly = false, Destructive = false, Idempotent = false)] [WarpHttpPost("/api/time-off/requests")] public sealed class RequestLeaveCommandHandler(...) { ... } ``` The rules an agent follows for every tool: 1. **Self-service.** A tool reads or changes the person's own records. Never mark an admin endpoint or one gated by a permission a person might hold over others. 2. **Data minimisation.** If an endpoint returns other people's data to some callers (a manager's view, a list by sequential id), do not mark it: add a `/api/my/...` sibling that filters on the caller and takes no subject. 3. **Authorization metadata.** The endpoint needs `[Authorize]` or a policy; startup refuses one without (unless you have a fallback policy). The call runs through the endpoint's own authorization as the user your resolver returned. 4. **The person's id is bound, not asked.** `BindFromCaller("userId", "app_user_id")` takes `userId` out of the tool's schema and fills it from the caller; an agent sending it is refused. A name no tool takes is warned at startup. 5. **Descriptions are for the agent:** what comes back, field by field; what each argument means; every refusal; for a write tool, "call it only when the person asked for this change". Beacon appends the route and the read/write flag. 6. **`ReadOnly` is always declared.** A write tool (`ReadOnly = false`) needs `AllowWriteTools`, is offered to signed-in people only, makes clients ask before each call, and is audited as `api_:write`. ### Recognising an agent call A tool call's principal has the authentication type `BeaconMcpUser`. Use it to hold agents to less than the browser — self-service only, whatever the person may do there: ```csharp if (httpContext.User.IsBeaconMcpCall()) { // no permissions, no manager grants: the person's own records only } ``` ### Refusals reach the agent as your problem details Tool calls run your `IExceptionHandler`s, as `UseExceptionHandler()` does for the browser: a refusal your application maps to a 403/404/400 problem reaches the agent as that status and body. An exception no handler takes is a generic error. ## 6. Entra app registration On **your application's existing registration** (the `AzureAd:ClientId` one): - [ ] Manifest: `api.requestedAccessTokenVersion = 2`. First — Entra only accepts an HTTPS Application ID URI on a v2 app. - [ ] Expose an API: keep `api://` and add the exact MCP URL `https:///beacon/mcp` to `identifierUris`. Claude sends it as the `resource`; a mismatch fails with **`AADSTS9010010`**. The domain must be a verified domain of the tenant (`*.azurecontainerapps.io` and similar cannot be). - [ ] Expose an API: add the delegated scope **`mcp.access`**. **A client registration for Claude** (Entra has no dynamic client registration): - [ ] New app registration, Web platform, redirect URI `https://claude.ai/api/mcp/auth_callback`. - [ ] A client secret (record its expiry: the connector stops when it lapses). - [ ] API permissions: your API's `mcp.access` (delegated) and `offline_access`; grant admin consent. - [ ] Give the client id and secret to whoever adds the connector in Claude; optionally list the client id in `McpAccess:AllowedClientIds`. ## 7. Test it ```csharp var tokens = new McpTestTokens(tenantId, clientId); var users = new McpTestUsers().Add(objectId: "oid-1", id: "42", displayName: "Ada", new Claim("app_user_id", "42")); builder.Services.TrustMcpTestTokens(tokens); // after AddBeaconMcpGateway, in the test host only builder.Services.UseMcpTestUsers(users); await using var client = await McpTestClient.ConnectAsync(app.GetTestClient(), tokens.For("oid-1")); var tools = await client.ListToolsAsync(); var result = await client.CallToolAsync("api_my_orders"); ``` For trying the gateway by hand on a laptop without Entra, run a local OIDC mock (for example `ghcr.io/navikt/mock-oauth2-server`) that mints delegated tokens with your tenant id, audience, an `oid` and `scp=mcp.access`, and point `AzureAd:Instance`/`TenantId` validation at it in Development only. The client is the official MCP SDK's, so a test goes through the bearer middleware, the caller mapping, the route policy, the MCP filters and audit, and your endpoint's own authorization. Listing and calling tools needs Beacon's database (project access and the audit live there): run those tests against a real PostgreSQL or SQL Server with Beacon's schema migrated. Pin, in your own tests: exactly which endpoints are tools (and which write), that each refuses somebody else's data, and that an agent call is self-service only. ## 8. Verify a deployment `beacon-doctor` walks the sign-in path the way an MCP client does and prints PASS/WARN/FAIL (exit code 1 on a failure): ```bash dotnet tool install -g Moberg.Beacon.Doctor beacon-doctor mcp https://your-app/beacon/mcp # the challenge, discovery, Entra metadata, absent documents beacon-doctor mcp https://your-app/beacon/mcp --token "$TOKEN" # plus initialize and tools/list as that person ``` By hand: - [ ] `POST /beacon/mcp` without a token → 401 with `WWW-Authenticate: Bearer resource_metadata=...`. - [ ] `/.well-known/oauth-protected-resource` → your v2 issuer, `api:///mcp.access`, `offline_access`, and a `resource` equal, character for character, to an `identifierUris` entry. - [ ] `/health` → `beacon-mcp-gateway` Healthy. - [ ] Claude connects, lists your tools, a read answers, and a write tool asks before it runs. - [ ] After an hour the connector refreshes silently. | Symptom | Cause | |---|---| | `AADSTS9010010` | The MCP URL is not an `identifierUris` entry (exactly), or the domain is not verified. | | 401 `invalid_token` | Wrong issuer or audience: check `AzureAd:TenantId` / `ClientId`, and that the token is v2. | | 403 at `/beacon/mcp` | The token is valid but not a caller: wrong scope or client, the person unknown to your resolver, or the host project not synced yet. The log has the reason with the caller hash. | | 503 with `Retry-After` | Entra's signing keys could not be fetched. | | Re-prompted to sign in every hour | `offline_access` missing from `scopes_supported` or not consented. | | A tool error "failed with an unexpected error" | The endpoint threw and no `IExceptionHandler` took it. | | Startup: "relies on '…' being its only authentication scheme" | Name your default: `AddAuthentication("YourScheme")`. | | Startup: "Register your IClaimsTransformation before calling AddBeaconMcpGateway" | Move your registration above the gateway's. | --- # Saved Query Tools > Expose approved, parameterized saved queries as named, read-only MCP tools such as q_loan_book_by_month(from, to), so agents and routines call reviewed, versioned SQL instead of carrying their own. Source: https://moberghr.github.io/beacon/mcp/saved-query-tools/ An agent that writes its own SQL for a recurring question gets a slightly different query every time. A **saved query tool** turns a reviewed Beacon query into a fixed MCP tool, such as `q_loan_book_by_month(from, to)`, that always runs the same approved SQL with the caller's arguments bound as database parameters. Routines become deterministic and cheaper: they call a tool instead of shipping `queries/*.sql` files or asking `ask` to regenerate SQL. ## Expose a query 1. Write the query as usual. Parameters are `{name}` placeholders in the step SQL, with a type (Number, DateTime, String) and a description. The description becomes the argument's description in the tool schema. A placeholder is `{identifier}` (letters, digits, underscore, starting with a letter or underscore); newly exposing a query whose placeholder has another shape is refused with the parameter's name, and editing the query to a valid placeholder fixes it. A tool that is already exposed keeps running and can be edited or disabled as stored. 2. Get a version **approved** (see [the approval rule](#which-version-runs)). With the approval workflow enabled, saving SQL changes submits a version for approval; an Approver (or an Admin) other than the requester approves it on **Approvals** (`/beacon/approvals`; see [Queries → Approval workflow](https://moberghr.github.io/beacon/features/queries/#approval-workflow)). 3. On the query's detail page, a **SecurityAdmin** (or an Admin) opens the **MCP tool** card, ticks *Expose as MCP tool*, sets a name and (optionally) a description, and saves. The card shows which approved version calls run, or why none can. While the approval workflow is on, publishing is a third pair of eyes: whoever requested or approved the version the tool runs cannot enable the tool, nor can a caller Beacon cannot name (an API key without an active owner), who could be either (`403`), unless the host allows self-approval (`ApprovalWorkflow.AllowSelfApproval`). The check is made when the tool is enabled: once published, the tool serves whichever version is approved next, whoever requested or approved that one. With the workflow off nobody approves a change, and enabling is not checked against approvals. The same through the API (the SecurityAdmin policy, see [Functional permissions](https://moberghr.github.io/beacon/security/authorization/#functional-permissions); anyone but an Admin publishes only a query they can read, `404` otherwise): ```http PUT /beacon/api/queries/42/mcp-tool Content-Type: application/json { "enabled": true, "name": "loan_book_by_month", "description": "Outstanding loan book per month for a date range." } ``` Rules the endpoint enforces: | Field | Rule | | --- | --- | | `name` | `^[a-z][a-z0-9_]{2,40}$`; the MCP tool is `q_`. Unique across live (non-archived) queries, so it is unique in every project and always names one query. Required to enable. Two concurrent saves of the same name get the same "already used" error (`400`), never a `500`. | | `description` | Optional, at most 1000 characters. Falls back to the approved version's description, then its name. | | `enabled: true` | Requires a runnable approved version whose parameters make a valid tool (see below), and, while the approval workflow is on, a signed-in caller with a user id who neither requested nor approved that version (`403` otherwise, unless the host allows self-approval; checked when enabling, not again when a later version is approved). Disabling is always allowed, by anyone who may publish; the name stays reserved while it is set. Disabling with a name another query already uses succeeds and keeps the query's stored name. | The query detail DTO (`GET /beacon/api/queries/{id}`) carries `mcpToolEnabled`, `mcpToolName`, `mcpToolDescription`, `mcpToolRunnableVersionNumber` and `mcpToolIssue`. `mcpToolIssue` explains why the query is not a tool anyone can call: no approved active version, a parameter shape that cannot be a tool, or **not visible in any project**: no single project contains all of the data sources its steps read, so no caller can ever see it. The last one does not block enabling (a data source may be added to a project later); it is reported so the tool does not silently stay invisible. ## Which version runs A tool runs **only** the query's active version, and only when an approval request for **exactly that version** was approved. In the workflow's strict mode (`ApprovalWorkflow.RequireApprovedVersionToRun`) the approval must also have recorded the version's content hash, which binds it to exactly the content the reviewer approved: an approval recorded before Beacon kept content hashes names the version but not what it approved, so strict mode does not run that tool until the version is approved again (restore it and approve the submission). Outside strict mode such an approval still counts, and with the workflow on the host logs a startup Warning naming those tools (at most 20, with their count). In strict mode the startup Warning counts instead the queries with a tool or a subscription whose active version is not approved for its content hash. Nothing else is callable: | Version state | Callable? | | --- | --- | | Active, its approval request Approved (in strict mode: for the version's content hash) | **Yes**. This is the only runnable version | | Active, approved before Beacon kept content hashes | Outside strict mode only (named in the startup Warning) | | Pending approval (a later edit) | No. The tool keeps running the approved version until the edit is approved | | Rejected, or superseded by a newer submission | No | | Active without an approval (edited while the approval workflow was off, or restored from history while it was off) | No | | Draft / no version | No | A query with no runnable version is **not listed** as a tool at all, even when enabled. The tool executes the approved version's **snapshot** (its stored steps and final query), not the live step rows, so an unreviewed change to the live query can never change what the tool runs. In practice: enable the approval workflow on hosts that use saved-query tools. It is off by default; turn it on in code with `options.EnableApprovalWorkflow(...)` in `AddBeaconServices` (see [Approvals](https://moberghr.github.io/beacon/security/approvals/)). With the workflow on, a version restore is submitted for approval like an edit, so restoring an older version never makes it callable without a new approval. Subscriptions follow the same rule in the workflow's strict mode (`ApprovalWorkflow.RequireApprovedVersionToRun`, see [Queries → Approval workflow](https://moberghr.github.io/beacon/features/queries/#approval-workflow)). A version is also not callable when its parameters cannot form a tool: a parameter without a placeholder, a name that is not a plain identifier, the reserved name `project_id`, or the same parameter declared with different types in different steps. The MCP tool card shows the reason. ## What the agent sees Each tool is listed with: - **name** `q_`, title (the query name) and the description, plus the approved version number; - an **input schema** built from the parameters: `number` for Number, an ISO 8601 `string` for DateTime, a `string` (max 4000 characters) for String; every parameter is `required` (the query model has no defaults) and `additionalProperties` is false; - annotations `readOnlyHint: true`, `destructiveHint: false`, `idempotentHint: true`, `openWorldHint: false`. Arguments are validated before anything runs. A missing or unknown argument, a non-number for a Number, or a date that is not ISO 8601 is a tool error. Dates with `Z` or an offset are normalized to UTC; a bare date (`2026-01-31`) is passed as written. The result is a markdown table plus structured content: ```json { "columns": [{ "name": "month", "type": "string" }, { "name": "principal", "type": "number" }], "rows": [{ "month": "2026-01", "principal": 1250000.00 }], "rowCount": 1, "truncated": false, "queryVersionId": 318, "versionNumber": 4 } ``` `truncated` is true when the result had more rows than the project's MCP row limit (or rows were dropped to keep the payload within 256 KB, then `rowsOmittedForSize` is also true). **Many tools.** Up to `SavedQueryTools.NamedToolLimit` tools (default 25) visible to one caller, each is its own `q_` tool. Above it, the caller gets two catalog tools instead: | Tool | Parameters | Returns | | --- | --- | --- | | `search_saved_queries` | `query` (string, keywords matched against names and descriptions; empty lists every tool), `limit` (integer, default 10, max 50) | names, descriptions and input schemas | | `run_saved_query` | `name` (string, required), `arguments` (object matching the query's input schema, including `project_id` when the schema lists one) | the same result as the named tool | Configure the limit in `AddBeaconServices(..., options => options.SavedQueryTools.NamedToolLimit = 40)`. The project brief (`get_context` with `format: "agents_md"`) lists the project's saved-query tools with their parameters, so an agent workspace's `AGENTS.md` knows they exist. The tools coexist with the built-in tools and with [host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/) (`api_`): all three are served side by side. ## Using one from a routine An agent routine that refreshes a dashboard's data calls the tool and writes the structured rows as JSON, with no SQL in the routine. The `mcp.callTool` call below stands for whatever MCP client the routine runs in: ```ts // routines/refresh-loan-book.ts const result = await mcp.callTool("beacon", "q_loan_book_by_month", { from: "2025-10-01", to: "2026-10-01", }); if (result.isError) { throw new Error(result.content[0].text); } const { rows, truncated, versionNumber } = result.structuredContent; if (truncated) { throw new Error("Loan book result was truncated; narrow the date range or raise the project's row limit."); } await writeFile( "dist/data/loan-book.json", JSON.stringify({ generatedFrom: `q_loan_book_by_month v${versionNumber}`, rows }, null, 2), ); ``` When the approved SQL changes, the routine picks up the new version on its next run without an edit, and the `versionNumber` it records says which reviewed SQL produced the file. ## Security model - **Authorization is the caller's projects.** A tool is visible in a project only when that project contains **every** data source the approved version reads, and only in the caller's projects where its [project access](https://moberghr.github.io/beacon/security/authorization/#project-access) lets it run saved SQL (`Run`; the projects an API key or mapped caller reaches are described in the [MCP Server guide](https://moberghr.github.io/beacon/mcp/server/#overview)): a query over a data source shared by projects A and B shows in both only if each contains all of its sources, and a caller of A never sees a query that needs a B-only source. A tool outside your projects answers exactly like one that does not exist. A caller with no project access sees no tools (fail closed). When your credentials reach the same tool through several projects, the schema offers `project_id` and the call requires it. - **Visibility follows data-source membership, not an owning project.** A saved query has no owning project: a tool appears in every project of the caller that contains all of its data sources, so linking a data source to another project can make an existing tool visible there. The tool's SQL is never listed, and the caller's `Run` permission is still checked in each project and again right before the query runs. - **Re-checked before it runs.** Right before any SQL runs, the call checks the steps it is about to run again: the caller may still run saved SQL in the project, and the project still links every data source those steps read. Otherwise the call fails ("not available in project N for your credentials") without running anything, and is audited. - **Execute scope.** `/beacon/mcp` requires the Execute scope, so a Read-scoped API key cannot call saved-query tools; project-restricted API keys work and are held to their project list. - **Read-only execution.** Every step's SQL, with its arguments bound, goes through the shared execution gate (regex guardrail, AST read-only validator, cross-database rule, system catalog rule, row limit) and runs through the provider's read-only path: a `READ ONLY` transaction on PostgreSQL and MySQL; on the other engines the parser gates and the read-only login you connect with (see [How a query runs](https://moberghr.github.io/beacon/features/queries/#how-a-query-runs)). A final query joins the steps in in-memory SQLite behind the same gate. - **Parameters, never interpolation.** Each `{placeholder}` is replaced by a generated database parameter (`@p0`, `@p1`, …) and the value is sent as a parameter; argument text never reaches the SQL. - **Host-managed sources.** When a step reads a [host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/) data source, the host's allow-list and column policy apply, in the gate and again in the provider, and masked columns come back masked. - **Row limits and PII.** The final result is capped at the project's MCP row limit. An intermediate step of a multi-step query may return at most 50,000 rows; past that the call fails instead of joining an incomplete set. PII columns are masked per the project's PII detection setting before rows leave a step. - **Audit.** Every call, whether it succeeds or fails and including unknown tools and invalid arguments, is written to the MCP audit log as `q_` (a search as `search_saved_queries`), with project, data source, row count and error. Arguments are kept only when the project's retention settings allow content, and are never written to the application log. The project is resolved **before** the arguments are validated, so an invalid-argument row follows that project's retention setting. When no project can be established (an ambiguous or foreign `project_id`, an unknown tool, a search across several projects) the row is structural only (tool, input size and error class, never argument content) rather than falling back to the global setting. - **No learning signals.** Saved queries are pre-approved SQL, not the caller's SQL, so they are **audit-only**: `McpSignalService` does not record them and they never feed pattern mining or golden examples. ## Known limits - Parameters have no defaults or allowed-value lists in the query model, so every argument is required and free-form within its type. - Only database data sources run as tools; API data source steps are rejected. - A multi-step query whose intermediate step returns no rows cannot join (the in-memory table is not created); the call fails with the SQLite error. - Column types in the structured result are inferred from the values returned; an empty result has no columns. --- # Host Endpoint Tools > Expose selected endpoints of the host application as MCP tools (read-only by default, write tools behind an explicit opt-in) invoked in-process as the caller through the endpoint's own model binding, filters and authorization policies. Source: https://moberghr.github.io/beacon/mcp/host-endpoint-tools/ A host application already has endpoints that answer the questions an agent asks: a back-office grid of a customer's loans, a lookup by id. `AddHostEndpointTools` turns the endpoints you mark into MCP tools. Each call runs **in-process, through the endpoint's real pipeline** (model binding, action filters, authorization policies), as a host identity built for the MCP caller, never as Beacon, and never with the caller's own token. Every exposed endpoint **declares** whether it is read-only; Beacon never infers it from the HTTP verb, because back offices read over POST (Kendo grids, `[FromForm]` filters). A read-only endpoint on any verb other than GET or HEAD is named in a startup Warning; set `AllowReadOnlyPost = false` to expose read-only tools on GET and HEAD only. By default only read-only endpoints are exposed; endpoints that change data become tools only when the host opts in with `AllowWriteTools` (see [Write tools](#write-tools)). ## Quick start ```csharp builder.Services.AddBeaconMcp() .AddHostEndpointTools(o => { o.ProjectName = "Back Office"; // the tools belong to this Beacon project o.MaxResponseBytes = 1_048_576; // default 1 MiB o.NamedToolLimit = 20; // above this: search_api + call_api o.RequestTimeout = TimeSpan.FromSeconds(30); // default 30 s o.MaxConcurrentCalls = 64; // default 64 calls in flight in total o.MaxConcurrentCallsPerCaller = 16; // default 16 per MCP caller o.QueueWait = TimeSpan.FromSeconds(5); // how long a call over a limit waits before it is refused // o.AllowReadOnlyPost = false; // default true; false: read-only tools on GET/HEAD only // o.TrustedAuthenticationSchemes = ["..."]; // default none; see "Authentication schemes" below // o.AllowWriteTools = true; // default false; see "Write tools" below // o.DescriptionNote = tool => "..."; // default Beacon's note; see "Description" below }); ``` `ProjectName` names the **host project**, the same one `ExposeDbContext` and `ExposeDocs` use. Beacon finds it by its host key (`Project.HostManagedKey = host:{name, lower-cased}`), never by its display name, so a project a Beacon user creates with the same name never gets the tools (see [Host documentation](https://moberghr.github.io/beacon/mcp/host-docs/#the-host-project)). The tools appear once the host has run `SyncBeaconHostAsync` and the project exists. Mark a controller action: ```csharp [HttpPost] [Authorize] // required: [Permission] is a filter, not authorization metadata [Permission(AdminPermission.ViewLoans)] // the host's own filter: runs in the action pipeline (needs a dispatch middleware, below) [BeaconTool("customer_loans", Description = "Loans of one customer, newest first.", ReadOnly = true)] public IActionResult CustomerLoans([DataSourceRequest] DataSourceRequest request, int customerId) { // ... return Json(loans.ToDataSourceResult(request)); } ``` Or a minimal API endpoint: ```csharp app.MapGet("/api/loans/{id:int}", (int id, LoanService loans) => loans.Find(id)) .RequireAuthorization("ViewLoans") .WithBeaconTool("loan_by_id", "One loan by id.", readOnly: true); ``` Or a handler class whose framework copies class attributes into its endpoint's metadata, such as a [Moberg Warp](https://moberghr.github.io/warp/) handler with a `[WarpHttp*]` route: ```csharp [Authorize] [BeaconTool("my_assets", Description = "The signed-in user's assets, newest first.", ReadOnly = true)] [WarpHttpGet("/api/my/assets")] public sealed class ListMyAssetsQueryHandler(AppDbContext db) : IRequestHandler> { // ... } ``` On an MVC controller class `[BeaconTool]` is refused at startup: it would mark every action under one name, so it goes on the action. The tool is exposed as `api_customer_loans` (or `api_loan_by_id`, `api_my_assets`). The endpoint must carry authorization metadata (`[Authorize]` on the action or controller, a policy, or `[AllowAnonymous]`), or the host must have a fallback authorization policy; otherwise startup refuses it. A permission check implemented as a filter (like the `[Permission]` filter above) is not metadata. A host that has neither `[Authorize]` on its controllers nor a fallback policy puts `[Authorize]` on every exposed action, next to `[BeaconTool]` and its own permission filter, as above. An exposed endpoint whose only requirement is an authenticated user (`[Authorize]` or `RequireAuthorization()` with no policy or roles, under the default policy, or a fallback policy that only requires authentication) is logged as a startup **Warning** naming it: every mapped MCP caller of the project can call it. It is still exposed, since a permission filter like `[Permission]` may check the rest; give it a policy or roles if nothing does. A tool that binds an argument from the caller ([Arguments the caller fills](#arguments-the-caller-fills)) is the self-service pattern and is not warned. ### Authentication schemes Beacon authorizes the dispatched call against the host principal it built for the MCP caller; it never runs an authentication handler. A policy that names **authentication schemes** (`[Authorize(AuthenticationSchemes = "StepUp")]`, `new AuthorizationPolicyBuilder("PartnerApi")`, a default or fallback policy with schemes) would therefore be evaluated as if that scheme had authenticated the caller: a step-up or MFA scheme would be bypassed. So an exposed endpoint whose combined policy names any scheme **fails startup**, naming the endpoint and the schemes, unless the host lists each scheme in `TrustedAuthenticationSchemes`: ```csharp .AddHostEndpointTools(o => { o.ProjectName = "Back Office"; o.TrustedAuthenticationSchemes = ["Cookies"]; // only if the MCP caller is an acceptable stand-in for it }); ``` List a scheme only when an authorized MCP caller really should count as authenticated by it. Never list a step-up, MFA or partner scheme. ## What gets exposed - Only endpoints carrying `[BeaconTool]` / `.WithBeaconTool(...)`. Everything else is invisible. - The name must match `^[a-z][a-z0-9_]{2,47}$` and be unique. `ReadOnly` must be declared: a missing flag **fails startup** with a message naming the endpoint, and so does `ReadOnly = false` unless `AllowWriteTools` is on (see [Write tools](#write-tools)). So do bad or duplicate names, `Destructive = true` on a read-only tool, file-upload parameters, `[AsParameters]`, an endpoint with no authorization at all, and an endpoint whose policy names an authentication scheme that is not in `TrustedAuthenticationSchemes` (see above). - `ReadOnly = true` on a verb other than GET or HEAD (GET is used when the endpoint takes several verbs) logs a startup Warning naming the endpoint. With `AllowReadOnlyPost` on (the default) it is still exposed; with it off it is not. A read-only endpoint on PUT, PATCH or DELETE most likely changes data: declare it a write tool. - Each tool name has one owner, checked once the pipeline is built. A name the saved-query tools (`q_`, `search_saved_queries`, `run_saved_query`, while saved-query tools are served) and the endpoint tools (the `api_` tools discovered, or `search_api` / `call_api` in catalog mode) both claim **fails startup**, naming it. An attribute tool the host adds with such a name only logs a warning: calls to that name reach the attribute tool. In `tools/list` the saved-query tools come before the endpoint tools (`AddBeaconMcp()` first, then `AddHostEndpointTools`). - Endpoints are discovered when the host's pipeline is built (they do not exist at DI time), from the `EndpointDataSource`. Parameters come from **ApiExplorer** when it describes the endpoint (`[ApiController]` controllers, or `AddEndpointsApiExplorer()` for minimal APIs), otherwise from the action's parameters and binding attributes. Swagger / Swashbuckle is never needed. - **Description**: the attribute's `Description`, else the action's XML doc `` (enable `GenerateDocumentationFile`; `` docs describe the arguments), else the endpoint's summary/description metadata, else a generic "Calls POST /route" line. Every description ends with a note naming the method and route and saying the tool runs with the caller's host permissions (a write tool's also says it changes data and is to be called only when the user asked). When the endpoints run with less than the caller's own permissions, or you word it otherwise, set `DescriptionNote`: it gets each tool's name, method, route and `ReadOnly` flag and returns the note, `null` or an empty string ends the description at the summary, and `tool.DefaultNote` is Beacon's own text: ```csharp o.DescriptionNote = tool => tool.ReadOnly ? $"Self-service endpoint {tool.HttpMethod} {tool.Route}: it sees only your own records." : tool.DefaultNote; ``` ### Arguments | Bound from | Tool argument | | --- | --- | | Route | one argument per route value, required unless optional in the template; URL-encoded into the path | | Query / form, simple types | one argument per value; collections as arrays (sent as repeated keys) | | Query / form, a model (`[FromForm] LoanFilter filter`, or an unannotated model on a plain MVC controller) | flattened: one argument per property (`CustomerId`, `Period.From`, up to 3 levels); collections of objects are skipped | | `[FromBody]` | one argument named after the parameter, its schema from `System.Text.Json`'s `JsonSchemaExporter` | | `[FromHeader]` | one argument named after the header. `Authorization`, `Cookie`, `Host` and content headers are refused | | A custom model binder (Kendo's `[DataSourceRequest]`) | one object argument whose members are sent as raw fields: `{ "page": 1, "pageSize": 20, "sort": "CreatedDate-desc" }` | | `CancellationToken`, `HttpContext`, `[FromServices]`, DI-injected minimal API parameters | not arguments | Unknown and missing required arguments are rejected before anything is dispatched. ### Arguments the caller fills An agent does not know the person's id in your application, so an endpoint that takes it (`/api/my/assets?userId=…`, a body with `RequestingUserId`) would otherwise need "leave it out, or pass 0, to mean me" logic. Bind such an argument to the caller instead: ```csharp builder.Services.AddBeaconMcp() .AddHostEndpointTools(x => { x.ProjectName = "Shop"; x.BindFromCaller("userId", "shop_user_id"); // a claim of the host principal the call runs as x.BindFromCaller("requestingUserId", "shop_user_id"); }); ``` On every tool, a route, query, header or form value with that name (compared ignoring case), or a top-level property of its JSON body, is left out of the input schema and of its required list, and the call carries the caller's value — a body property written as the type its schema declares. An agent that sends a bound name anyway — as an argument, as a raw field of a custom-binder model, or as a body property in any casing — is refused, on every tool, since a custom binder or a dictionary could still read it. A principal without a value refuses the call, and a bound name that no tool takes is warned at startup (most likely a typo). A `Func` overload reads the value any other way. Your endpoint still checks ownership as it does for the browser. ### Named tools or a catalog Up to `NamedToolLimit` tools, each is its own MCP tool. Above it, agents get two tools instead, so a large host does not flood the tool list: `search_api` (keywords → matching tools with their input schemas) and `call_api` (a name plus arguments). Read-only tools carry `readOnlyHint = true`, `destructiveHint = false`, `idempotentHint = true`, `openWorldHint = false`; write tools carry the hints in [Write tools](#write-tools). `search_api` is always read-only. `call_api` is read-only only while every tool it can reach for this caller is; once it can reach a write tool it carries `readOnlyHint = false` (and `destructiveHint = true` if any of them is destructive), so clients confirm every `call_api` call. Each `search_api` result carries `read_only` (and, for a write tool, `destructive` and `idempotent`). The eight built-in tools are unaffected. ## Write tools A host can let agents change data through its own endpoints: an HR app letting a user book or withdraw their own leave from Claude Desktop. This is off unless the host opts in, and each endpoint still declares itself: ```csharp .AddHostEndpointTools(o => { o.ProjectName = "Employee Portal"; o.AllowWriteTools = true; // without it, ReadOnly = false fails startup }); [HttpPost("leave")] [Authorize(Policy = "Employee")] [BeaconTool("book_leave", Description = "Book leave for yourself.", ReadOnly = false)] public IActionResult BookLeave([FromBody] BookLeaveRequest request) { ... } [HttpDelete("leave/{id:int}")] [Authorize(Policy = "Employee")] [BeaconTool("withdraw_leave", Description = "Withdraw one of your leave requests.", ReadOnly = false, Destructive = true, Idempotent = true)] public IActionResult WithdrawLeave(int id) { ... } // Minimal API: .WithBeaconTool("withdraw_leave", "...", readOnly: false, destructive: true, idempotent: true) ``` - **Opt-in.** `AllowWriteTools` (default `false`) is the host-level switch. `ReadOnly` must still be declared on every endpoint; Beacon never infers it from the verb, in either direction. - **Annotations.** A write tool carries `readOnlyHint = false`, `destructiveHint = Destructive` (default `false`), `idempotentHint = Idempotent` (default `false`) and `openWorldHint = false`. Declare `Destructive = true` for an endpoint that deletes or irreversibly overwrites data. Its description says it changes data and should be called only when the user asked for the change. - **Confirmation is the client's.** `readOnlyHint = false` is what makes MCP clients (Claude Desktop, Claude Code) ask the user before running the tool. Beacon does not add its own confirmation step: MCP elicitation is not used for write tools, so a client that auto-approves tools runs them without asking. - **User callers only.** A write tool is listed to, and runs for, a mapped Entra **user** caller only. A **system** caller (an app-only token or service user) never sees write tools (not in the tool list, not in `search_api`) and a call to one is refused with `Forbidden: api_ changes data in the host application, and write tools run only for a signed-in user; a system caller cannot call them.`, whatever `HostClaims` it holds. A system identity acts with standing permissions and has no person who asked for the change and no client to confirm it. API-key callers cannot call any endpoint tool. - **The endpoint is the authority.** Beacon checks the caller, the project and the endpoint's authorization policy, exactly as for reads, and adds nothing else. The endpoint must do its own input validation, business rules and record-level authorization ("only your own leave") against the host principal your `IMcpHostPrincipalFactory` builds: treat a tool call like any other request from that user. Antiforgery passes the same way as for reads (see [Antiforgery](#antiforgery)), so it is not a defence here. - **Audit.** Write calls are audited under the tool name `api_:write` (see [Audit and privacy](#audit-and-privacy)). `IBeaconToolMetadata.Destructive` and `Idempotent` are default interface members: a host's own implementation of the interface keeps compiling and behaves as `false` for both. ## Who the call runs as Endpoint tools need a **mapped MCP caller**, an Entra user or system identity from [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/), configured for the tools' project and allowed to read it by its [project access](https://moberghr.github.io/beacon/security/authorization/#project-access): a user caller reaches a restricted project only while its Beacon user is a member (see [What a caller reaches](https://moberghr.github.io/beacon/mcp/entra-callers/#what-a-caller-reaches)). API-key callers and callers outside the project neither see nor can call them (fail closed); neither can anyone while the project does not exist. `IMcpHostPrincipalFactory` builds the host principal from the caller: - **System caller** → identity `BeaconMcpSystem` with the caller's configured `HostClaims` plus its name. - **User caller** → identity `BeaconMcpUser` with name, email, preferred username and object id copied from the MCP principal, plus any `HostClaims` the caller mapper assigned (none with the configured mapper). The default gives a user **no host permissions**, and deliberately does **not** copy Entra `roles` or `groups`: a host role that happens to share a name (`Admin`) would otherwise be granted implicitly. Map users to host permissions in your own factory. The name, email and preferred username are **display data**: Entra lets a tenant change them, and a guest's email comes from its home tenant, so never map a user to host permissions by one of them. Use the object id (`oid`) together with the tenant id (`tid`). - `null` → the call is refused. A host whose permissions live in its own user store registers its own factory. The example below maps the Entra user to the host's admin user and its `AdminPermission` set. The host's permission claim is `HostClaimTypes.Permission` (`http://schemas.example.com/claims/permission`) and its value is the permission's **integer** value (`AdminPermission.ViewLoans.ToValueString()` → `"1007"`), which is what the host's `Identity.HasPermission` compares against. The admin lookup is a **placeholder**: use whatever service the host already has for loading an active admin and their permissions (it stores the Entra tenant id and object id on the admin record): ```csharp builder.Services.AddSingleton(); internal sealed class HostPrincipalFactory(IServiceScopeFactory scopes) : IMcpHostPrincipalFactory { public async Task CreateAsync(McpCaller caller, ClaimsPrincipal mcpPrincipal, CancellationToken ct) { if (caller.Kind == McpCallerKind.System) { // HostClaims come from Beacon:Mcp:Callers:Systems[*]:HostClaims: keep the set minimal. return new ClaimsPrincipal(new ClaimsIdentity(caller.HostClaims, "BeaconMcpSystem")); } // Key the lookup on the immutable object id (and the tenant id), never on email, UPN or preferred username. var objectId = mcpPrincipal.FindFirst("oid")?.Value; var tenantId = mcpPrincipal.FindFirst("tid")?.Value; await using var scope = scopes.CreateAsyncScope(); // PLACEHOLDER: your own lookup of an active admin user and their AdminPermission set. var admins = scope.ServiceProvider.GetRequiredService(); var admin = objectId == null || tenantId == null ? null : await admins.FindActiveAdminByEntraIdAsync(tenantId, objectId, ct); if (admin == null) { return null; // not an admin: refused } var claims = admin.Permissions // IEnumerable .Select(x => new Claim(HostClaimTypes.Permission, x.ToValueString())) .Append(new Claim(ClaimTypes.Name, admin.UserName)); return new ClaimsPrincipal(new ClaimsIdentity(claims, "BeaconMcpUser")); } } ``` System identity host claims, in configuration (the value is the integer `AdminPermission` value; `1007` is `ViewLoans`): ```json "Beacon": { "Mcp": { "Callers": { "Systems": [ { "Name": "routine-runner", "ClientId": "", "ProjectIds": [ 3 ], "HostClaims": [ { "Type": "http://schemas.example.com/claims/permission", "Value": "1007" } ] } ] } } } ``` ## How a call is dispatched 1. The caller, project and arguments are checked; the host principal is built. The call then takes a slot: at most `MaxConcurrentCallsPerCaller` calls per MCP caller and `MaxConcurrentCalls` in total are in flight. A call past either limit waits up to `QueueWait` (5 s) for a slot, then is refused ("Too many host endpoint calls are in progress; retry shortly.") and audited. The slot is held until the endpoint really ends, so an endpoint that ignores its `RequestAborted` token keeps it after a timeout. 2. A **new DI scope** and a new `HttpContext` are created. The request carries the method, the route filled from the endpoint's route pattern, the query string, the declared headers only, and a JSON or `application/x-www-form-urlencoded` body. **Nothing of the MCP request is forwarded**: no `Authorization` header, no cookies, no connection info. It also sends an `Accept` header preferring JSON (`application/json, text/plain;q=0.9, */*;q=0.1`) and `X-Requested-With: XMLHttpRequest`. 3. The endpoint and route values are set as routing would set them, and checked as routing would check them first: a value that fails the route's constraints (`{id:int}`, `{slug:regex(...)}`, a custom `IRouteConstraint`, evaluated without an `HttpContext`), a `/` in a single-segment value, or a `.` or `..` segment in any route value is refused as invalid arguments before anything runs. 4. **Authorization** runs before the endpoint, exactly as the authorization middleware would, against the host principal: the endpoint's `[Authorize]` / policy / requirement metadata combined through the host's `IAuthorizationPolicyProvider`, evaluated with `IPolicyEvaluator`. `[AllowAnonymous]` is honoured. No authentication handler runs, which is why a policy naming an untrusted scheme is refused at startup (see [Authentication schemes](#authentication-schemes)). An endpoint without authorization metadata falls back to the host's `FallbackPolicy`; with no fallback policy either it is refused at startup (and at dispatch). A denial is the tool error `Forbidden.` with no detail. An endpoint marked `[AllowAnonymous]` is exposable and runs without an authorization check: the host itself declared it public, so exposing it to an authorized MCP caller grants nothing new. 5. The registered `IHostEndpointDispatchMiddleware` chain runs, in registration order, around the endpoint (see below). 6. The endpoint's `RequestDelegate` runs; for MVC that is the whole action pipeline: model binding, validation, authorization/action/exception filters, the result. 7. The response is captured into a buffer capped at `MaxResponseBytes`; the request is aborted after `RequestTimeout` or when the MCP call is cancelled. Aborting signals `RequestAborted`; an endpoint that ignores it keeps running, with its DI scope and its call slot, until it ends, so slow endpoints that do not flow their cancellation token are bounded by the concurrency limits, not by the timeout. Host code that reads `IHttpContextAccessor` sees the dispatched request. The dispatch runs in its own execution context, so the MCP request's context (and the audit identity) is untouched. ### Host middleware and ambient state: dispatch middleware Host **middleware does not run** on dispatch: only the endpoint's own pipeline. When the endpoint depends on state a middleware sets, register an `IHostEndpointDispatchMiddleware` (namespace `Beacon.Core.HostEndpoints`, shipped in `Moberg.Beacon.MCP`): ```csharp Task InvokeAsync(HttpContext syntheticContext, McpCaller caller, Func next, CancellationToken cancellationToken); ``` Registered middlewares are chained in registration order, with the endpoint's request delegate as the innermost `next`. The chain starts after `User`, `RequestServices`, the endpoint and route values are set and after Beacon's authorization check. As in ASP.NET Core middleware, ambient state set before `await next()` (`Thread.CurrentPrincipal`, an `AsyncLocal`) is visible to the endpoint and its filters; async work before it is fine. A middleware that does not call `next` refuses the call; one that throws fails it. Both return a generic tool error and are audited. The dispatch runs in its own execution context, so the MCP request's own ambient principal is never changed. Take a host whose `[Permission(AdminPermission.X)]` is an `IAsyncAuthorizationFilter` that checks `Identity.Current.HasPermission(...)`, where `Identity.Current` reads `Thread.CurrentPrincipal`, which the host's own middleware sets with `Identity.Set(identity)`: it takes a `ClaimsIdentity`, not a principal. Without a dispatch middleware every tool call would be denied. The factory above maps the Entra user to admin permissions; the middleware makes that principal ambient: ```csharp builder.Services.AddSingleton(); internal sealed class HostIdentityMiddleware : IHostEndpointDispatchMiddleware { public async Task InvokeAsync(HttpContext ctx, McpCaller caller, Func next, CancellationToken cancellationToken) { // Identity.Set takes a ClaimsIdentity: the identity of the principal IMcpHostPrincipalFactory built. Identity.Set((ClaimsIdentity)ctx.User.Identity!); await next(); } } ``` ### Antiforgery For a call with a body (a write tool, or a read-only tool over POST), Beacon **mints a real antiforgery token pair** for the host principal with the host's own `IAntiforgery` and puts it on the dispatched request (the cookie token as a cookie, the request token in the configured header, or the form field when the header is disabled). The host's own validation (`[ValidateAntiForgeryToken]`, a global `AutoValidateAntiforgeryTokenAttribute`, minimal API form endpoints) therefore runs and passes. For minimal API endpoints that require validation, Beacon validates the request the way `UseAntiforgery()` does and records the result. Nothing is exempted and real browser requests are validated exactly as before: the tokens are minted server-side and never leave the process (CSRF cannot apply to a request that never came from a browser). A host with a custom anti-CSRF scheme (a bespoke header check, an `IAntiforgeryAdditionalDataProvider` that inspects the request) must make sure it accepts these requests. ### Responses - 2xx JSON → the JSON as text plus `structuredContent`. MCP structured content must be an object, so a top-level JSON array is wrapped as `{ "items": [...] }` (the text content is the unwrapped JSON). - 2xx text → text. An empty body → a "no content" line. - Any other content type → a tool error. - Non-2xx → a tool error with the status and a short body excerpt. - An exception in the endpoint → your `IExceptionHandler`s run, in registration order, as `UseExceptionHandler()` runs them for a browser request (host middleware does not run for a dispatched call). The response the first one writes — the 403, 404 or 400 problem your application maps a deliberate refusal to — is the tool error, its body passed to the agent as the handler wrote it. A handler that takes the exception without setting a status answers 500. An exception no handler takes is a generic tool error (the message is not returned or logged). Set `UseHostExceptionHandlers = false` to always answer the generic error. - Over `MaxResponseBytes` → a tool error; narrow the request. ## Audit and privacy Every call (success, forbidden, refused, invalid arguments, timeout, error) writes an MCP audit row with the tool name `api_` (or `search_api`), the project, the duration, the caller kind and hash, and the outcome. A call to a **write tool** is recorded as `api_:write` (whether it was called by name or through `call_api`, and also when it was refused) so writes can be filtered in the audit log (`tool=api_book_leave:write`) without a schema change. Metrics still bucket it under `api_*`. The arguments are stored only as the project's content-retention settings allow: under the [retention lock](https://moberghr.github.io/beacon/mcp/server/) they are reduced to their size. Argument values and response bodies are never written to logs. `get_context format=agents_md` lists the project's endpoint tools in the brief (name and first description line): the ones the caller's tool list offers, as the overview does. Write tools are marked *(writes)*. ## Limits - Write tools are off unless `AllowWriteTools` is on, run for user callers only, and rely on the client's tool approval for confirmation (no Beacon-side approval or elicitation step). - Host **middleware does not run**: only the endpoint's own pipeline. Recreate what the endpoint needs from it (an ambient principal, a tenant, a culture, an admin user loaded into `HttpContext.Items`) in an `IHostEndpointDispatchMiddleware`. A global MVC `AuthorizeFilter` whose policy names authentication schemes re-authenticates by scheme, finds no cookie and denies; express such rules as endpoint metadata (`[Authorize]` / policy attributes) instead. - Endpoint metadata that names an authentication scheme fails startup unless the scheme is in `TrustedAuthenticationSchemes`. - File uploads and downloads are not supported; collections of objects in form models are not flattened. - The request host is `localhost` and there is no remote IP address; do not rely on either for authorization. - Parameter names from custom binders are not known, so their fields are free-form. --- # Host DbContext > Expose the host application's own EF Core DbContext as a read-only Beacon data source: default-deny table allow-list, column exclusion and masking, no stored connection string. Source: https://moberghr.github.io/beacon/mcp/host-dbcontext/ When Beacon is embedded in an existing .NET application, that application's own database can be exposed to Beacon (and through it to MCP agents) straight from its EF Core model. No connection string is stored, the metadata comes from the model, and only the tables you allow-list can be read. ## Quick start ```csharp var beacon = builder.Services.AddBeaconServices(builder.Configuration, o => { /* as today */ }); beacon.ExposeDbContext(o => o .ReadOnlyConnection("BeaconReadOnly") // ConnectionStrings:BeaconReadOnly: a read-only SQL login .AllowTables("Loan", "LoanInstallment", "Customer") .AllowEntity() .ExcludeColumns(x => x.Table == "Customer" && x.Name == "KycNotes") .MaskColumns(x => x.Name is "SSN" or "AccountNumber")); beacon.AddSqlServerConnector(); beacon.UseSqlServer(); // ends the builder chain: call ExposeDbContext first var app = builder.Build(); // ... apply migrations ... await app.Services.SyncBeaconHostDataSourcesAsync(); // creates / refreshes the data source ``` `SyncBeaconHostDataSourcesAsync` is a plain startup call (not a hosted service). It is idempotent: it creates the project (`ProjectName`, default = the data-source name), the data source and the project link on first run, and on later runs rewrites metadata only when the hash of the exposed model slice changed. When the host also ships documentation with [`ExposeDocs`](https://moberghr.github.io/beacon/mcp/host-docs/), call `SyncBeaconHostAsync()` instead: it runs this data-source sync and then imports the documents, as the single startup entrypoint. `SyncBeaconHostDataSourcesAsync` keeps working on its own. ## Options | Option | Meaning | |---|---| | `Name` | Data-source name. Default: the context type name. Also the stable key (`efcore:{Name}`). | | `ProjectName` | Project to create or attach to. Default: `Name`. | | `Description` | Description used when the project is created. | | `ReadOnlyConnectionStringName` / `ReadOnlyConnection(name)` | **Required.** Name of a `ConnectionStrings` entry. Validated at startup; resolved from configuration at execution time; never persisted. | | `Engine` | Normally inferred from the EF provider. Only SQL Server and PostgreSQL are supported. | | `AllowTables("Table", "schema.Table")` | Allow-list by store name (case-insensitive). Unknown names fail startup. | | `AllowEntity()` | Allow-list the table or view an entity maps to. | | `AllowAllTables()` | Expose everything. Deliberately explicit; prefer an allow-list. Logged as a warning at every sync. | | `ExcludeColumns(Func)` | Invisible in metadata **and** rejected at execution. | | `MaskColumns(Func)` | Visible in metadata (flagged `[PII: values are masked in query results]`); values masked in every result. Exclusion wins. | | `IncludeSecretLikeColumn("Table.Column")` | Lifts the secret-like hard-exclude for one column. Logged as a warning at every sync. | | `AllowFunctions("fn", …)` | Allows extra functions (unqualified name, case-insensitive) on top of the built-in allow-list. Only name functions that read nothing but their arguments. Catalog / settings / file / remote functions (`pg_*`, `xp_*`, `sp_*`, `fn_*`, `OPENROWSET`, `query_to_xml`, `row_to_json` …) are refused at startup. | `HostColumn` gives `Schema`, `Table`, `Name` (column), `ClrType`, `EntityType` and `PropertyName`. ### Secret-like hard-exclude Regardless of your predicates, a column is excluded when: - its column or property name **contains** `password`, `passwd`, `passphrase`, `passcode`, `secret`, `token`, `apikey`, `api_key`, `privatekey`, `private_key`, `credential`, `connectionstring`, `securitystamp` or `recoverycode` (also with an underscore: `connection_string`, `security_stamp`, `recovery_code`); - it has a **whole word** (camelCase / snake_case) `pin`, `pwd`, `salt`, `hash`, `otp`, `totp`, `hotp`, `mfa`, `refresh`, `pfx`, `nonce`, `seed`, `cvv` or `cvc`, so `PasswordHash`, `PinCode`, `OtpSeed`, `TotpKey` and `RefreshToken` are excluded while `Shipping` is not; - it has a `key` word beside a word naming key material (`api`, `access`, `secret`, `private`, `signing`, `encryption`, `authenticator`, `certificate`, `cert`, `master`, `license`, `hmac`, `aes`, `rsa`, `pgp`, `gpg`, `ssh`): `EncryptionKey` and `SigningKey` are excluded, `ProductKey` is not; - its **table** holds credentials: `AspNetUserTokens`, `DataProtectionKeys`, or any table whose name ends in the word `Keys` or `Tokens` (`ApiKeys`, `signing_keys`, `RefreshTokens`) — every column of it; - its entity is ASP.NET Core Identity's `IdentityUserToken` or Data Protection's `DataProtectionKey` (or derives from one), whatever its table is called — every column of it. Business words that only sometimes name a secret (`SigningDate`, `CertificateNumber`, `RecoveredAmount`, `TimeStamp`, `ConcurrencyStamp`) are not excluded on their own. Only `IncludeSecretLikeColumn` lifts the exclusion. Every sync logs the columns it excluded (`schema.Table.Column`). The rule is a best-effort name heuristic: also deny the secret columns to the read-only login. ## Metadata Read from the EF design-time model without opening a connection: tables and views (`ToTable` / `ToView`), schemas, column names and store types, nullability, max length, primary keys, indexes, and **foreign keys** (when the principal table is also exposed; they feed the schema-relationship graph and join suggestions). Descriptions come from `HasComment`, falling back to XML documentation (``) when the entity assembly ships its `.xml` file (`true`). Owned types and table splitting are merged into their table; entities mapped to SQL queries or functions are skipped; keyless views are included when allow-listed. A live catalog scan and sample-value collection are never run for a host data source; "Refresh metadata" in the UI is refused; restart the host (or call the sync again) instead. ## Security model - **No stored secret (§1.1).** `EncryptedConnectionData` holds only an encrypted reference, `{"hostConnectionStringName":"BeaconReadOnly"}`. The real string is read from `IConfiguration` whenever a connection is opened (`IDataSourceConnectionResolver`, the single decrypt-and-resolve point). - **Host-owned.** The data source is marked `IsReadOnly`, cannot be edited or deleted through the REST API/UI (the request is refused), and cannot be used by migration jobs. Descriptions, glossary, golden queries and the knowledge graph work as for any other data source. - **Policy enforced at execution.** Every SQL statement against a host data source is checked against the exposed slice, in the execution gate (MCP `query`, `dry_run`, cross-source, eval; a violation blocks regardless of `BlockOnSchemaFailure`) **and** again in the database provider (every path: `ask`, query previews, value grounding, documentation sampling), in saved-query steps and in data-quality rules. The check parses the SQL in the data source's own dialect and resolves every name the way the database does: - only a single `SELECT`; every table must be allow-listed **and schema-qualified** (`dbo.Customer`, `public."Customer"`), an unqualified name is rejected with `Qualify the table: dbo.Customer`, because the connection's real default schema / `search_path` may differ from the model's. Three-part / linked-server names, table-valued functions, `OPENROWSET` / `OPENJSON`, `SELECT INTO`, `FOR JSON/XML`, locking clauses, `NATURAL` joins and temporal `FOR SYSTEM_TIME` are rejected; - identifiers compare as the engine compares them (PostgreSQL: unquoted names fold to lower case, quoted names are exact; SQL Server: case-insensitive), for tables, CTE names, aliases and columns alike, so a CTE never hides a real table (`WITH "AuditLog" AS (…) SELECT … FROM auditlog` is rejected); - on PostgreSQL, every identifier is read as the server names it: an identifier longer than 63 bytes in UTF-8 is shortened to 63 bytes on a character boundary, as PostgreSQL silently does (quoted names too, output aliases included), and compared and masked by that name. Only what the server would read differently than written is rejected: two different identifiers of the statement that are the same in their first 63 bytes (PostgreSQL would read them as one; use names that differ sooner, or, when the host's own names collide, expose a view or rename the column on the host), and an unquoted identifier with a non-ASCII letter that changes case, such as `Šifra` or the Kelvin sign (PostgreSQL folds only `A`–`Z`; write it in lower case or quote it, `"Šifra"`); - names resolve with per-`SELECT` scopes: an unqualified column must be an exposed column of a relation in that `SELECT`'s own `FROM`; a correlated (outer) reference must be qualified with the outer alias; output aliases are visible only to the same `SELECT`'s `ORDER BY` (and PostgreSQL's `GROUP BY`) and to outer queries through a derived table's / CTE's columns. A bare relation name is never a column, so whole-row references (`SELECT c FROM … c`, `row_to_json(c)`, `(c).col`) are rejected; unknown and excluded columns are rejected anywhere; a table alias may be used only once per query; - on PostgreSQL a bare `GROUP BY` name is read the way the server reads it: as a column of the `SELECT`'s own relations first, and as an output alias only when none of them has a column of that name. A table can have columns the policy excludes or the model does not map, so an output alias in `GROUP BY` is accepted only when the `SELECT` reads derived tables or CTEs alone; over a table, group by the expression itself or by its position (`GROUP BY 1`); - on PostgreSQL, catalog types (`regclass`, `regrole`, `regproc`, `regtype` … and their arrays, `regclass[]` or `_regclass`, in any case, quoted or schema-qualified) are rejected as casts (`CAST(… AS regclass)`, `'…'::regclass`) and as typed literals (`regclass '…'`), and a string where a name belongs (`SELECT col 'x'`, which PostgreSQL reads as a typed literal) is rejected — quote names with double quotes; - `SELECT *` / `t.*` is rejected with a list of the exposed columns to use instead; - functions must be on a per-dialect **allow-list** of built-in string, date/time, math, conditional, conversion, aggregate, window and (on column values) JSON functions, plus any `AllowFunctions` entries; everything else, user-defined functions included, is rejected. Bind parameters are `@p0`, `@p1` … on both engines (the bare PostgreSQL `@` operator is rejected; use `abs()`), and a statement never runs with a parameter unbound; - string-building functions take only bounded literals, so one call cannot build a huge value: - the count of `repeat` (`REPLICATE` on SQL Server) and the width of `lpad` / `rpad` must be a number literal of at most 10,000, optionally quoted (`'5'`) or with one cast straight over it to an integer or numeric type (`'5'::int`, `CAST(5 AS bigint)`); a column, parameter, expression, a cast to another type or a cast of a cast is rejected, and a negative number passes, since it builds nothing; - the string `repeat` / `REPLICATE` repeats must be a plain string literal of at most 100 characters, so a column or a nested call cannot multiply the length (`lpad` / `rpad` cut their string to the width, so theirs is free); - PostgreSQL's `format()` needs a string-literal format string whose field widths add up to at most 10,000 and that uses sequential specifiers only (`%s`, so each argument is used once; a positional `%1$s` and a `*` width, taken from an argument, are rejected); - a call that can lengthen its input many times — `repeat` / `REPLICATE`, `replace` / `REPLACE`, `regexp_replace` and a `format()` that quotes (`%I` / `%L`) — may not appear anywhere inside an argument of another such call, so lengths add up instead of multiplying (`replace(replace(…))` is rejected; one `replace(col, ',', ';')`, and `lpad`, `left`, `substring` … around or inside it, pass); - SQL Server's `FORMAT` needs a string-literal format string with no number above 999 (a precision such as `N999999999` is rejected). Other costly work (regular expressions, large joins and aggregates, a value built up step by step through CTEs) is bounded by the statement timeout, the row cap and the database's own limit on one value (1 GB on PostgreSQL), not by the check — give the host login a server-side statement timeout too; - a masked column may appear **only** as a direct projection (optionally aliased, through CTEs, derived tables and `UNION ALL` the mask follows the value to its output key), inside `COUNT(col)`, or in `col IS NULL` / `col IS NOT NULL`. Any other use (`WHERE`, `JOIN … ON`/`USING`, `GROUP BY`, `ORDER BY` (by name, alias or ordinal), `DISTINCT`, `UNION`/`INTERSECT`/`EXCEPT`, window specifications, `COUNT(DISTINCT …)`, any other function, `CASE`, casts, or a subquery used as a value) is rejected, so masked values cannot be probed. Host-masked values are masked **fully** in every result (`***`; `NULL` stays `NULL`). The result is masked by **output position**, not by column name: the check knows which columns of the statement carry a masked value, and the provider masks those columns of the result whatever the database calls them: under any alias, a repeated name or another case. Names serve only as a consistency check that fails closed: a result column the database names differently than the check derived is masked too (logged as a warning with the data source id and counts), and a result whose column count differs from the statement's is discarded (`The result's columns did not line up with the columns the host data source policy checked, so the result was discarded.`). Every path that runs SQL on a host data source reads it through the provider: previews (paged ones included), subscriptions and their attachments, MCP tools, saved-query tools and the rows the in-memory join reads, so all of them return the masked values. - **Errors stay generic.** When a statement fails on the host database the caller gets `Query failed on the host database.` and the log records only the exception type and the driver's error code; server messages can quote column values. - **Read-only login.** SQL Server has no read-only transaction mode, so `SupportsDatabaseReadOnlyEnforcement` stays `false`; the parser gates plus the dedicated read-only login are the guarantee. On PostgreSQL the `READ ONLY` transaction backstop applies as usual. - **Login check at startup.** After every host data source is synced, Beacon asks each host database whether its read-only login can write, with fixed catalog statements outside the SQL gate (the exposed tables' names are parameters): - **PostgreSQL:** whether the login is a superuser (`rolsuper`) or holds `INSERT`, `UPDATE` or `DELETE` on an exposed table (`has_table_privilege`), including column-level `INSERT` or `UPDATE` (`has_any_column_privilege`). - **SQL Server and Azure Synapse:** whether the login is in `db_owner` or `db_datawriter`, holds a database-wide `INSERT`, `UPDATE` or `DELETE` permission, or holds one of them on an exposed table (`HAS_PERMS_BY_NAME(..., 'OBJECT', ...)`); the database-wide probes name the current database (`DB_NAME()`). Only those privileges are asked about (not `TRUNCATE`, DDL or other schemas' tables). A login that can write gets one Warning per data source naming it by id, never the connection string or the login. A check that cannot run, an answer the database does not give, or a source none of whose exposed tables the database finds logs that the login could not be verified: nothing checked is never reported as read-only. A connection the transport policy refuses is logged as not checked, and a disabled source is not checked at all (it makes no connection). The checks run in parallel, each bounded at 10 seconds, so startup waits about 10 seconds at most; they never fail or block startup. ### Behaviour to know - Beacon knows the columns of the EF model. A statement that references a column the model does not have is rejected. Grant the read-only login `SELECT` only on the allow-listed tables (and `DENY` secret columns) so the database applies the same boundary. - A function added with `AllowFunctions` runs with the read-only login's rights. Grant the login `EXECUTE` only on the functions you allow. - A masked column still answers `IS NULL` and `COUNT(col)`. Exclude the column instead when its presence must stay hidden too. - Identifiers are compared case-insensitively, also on case-sensitive SQL Server collations; a name whose case does not match fails on the server. - These forms are refused with a validation message: unqualified correlated references, the same alias in an inner and an outer query, `NATURAL` joins, `DISTINCT` or set operations over masked columns, and on PostgreSQL two identifiers that share their first 63 bytes or an unquoted identifier with a non-ASCII letter that changes case. - The 63-byte rule assumes a stock PostgreSQL build (`NAMEDATALEN` 64) and counts UTF-8 bytes. SQL Server keeps names up to 128 characters as written, so no length rule applies there. :::caution[Upgrading] Host column masking now works by output position. On PostgreSQL host data sources, a statement with two identifiers that are the same in their first 63 bytes, or with an unquoted identifier that has a non-ASCII letter that changes case, is now refused with a validation message: use names that differ sooner, write the name in lower case or quote it. A single identifier longer than 63 bytes is accepted and read as PostgreSQL shortens it. Check saved queries, subscriptions and saved-query tools on PostgreSQL host data sources for such aliases. A result column the database names differently than the check derived is now masked (the MCP `query` tool notes how many), and a host data source result whose columns do not match the statement now fails instead of being returned. ::: ## Example: restricted tables and masked columns ```csharp beacon.ExposeDbContext(o => { o.Name = "Shop"; o.ProjectName = "Shop Back Office"; o.ReadOnlyConnection("BeaconReadOnly") .AllowTables("Loan", "LoanInstallment", "Merchant", "Customer") .ExcludeColumns(x => x.Table == "Customer" && x.Name.StartsWith("Kyc")) .MaskColumns(x => x.Name is "SSN" or "AccountNumber" or "Email" or "Phone"); }); ``` `User.Secret`, `ProviderIntegrationKey.SecretKey` and `Customer.PartnerAuthorizationToken` are excluded by the hard-exclude even if their tables are allow-listed; the national-id (`SSN`) columns are visible to the agent as PII-flagged columns and masked in every result. --- # Host Docs > Ship the host application's markdown documentation with it and import it into a Beacon project. The documents are searchable by MCP agents with or without embeddings, readable through get_documentation, and summarised in an AGENTS.md project brief. Source: https://moberghr.github.io/beacon/mcp/host-docs/ A host application often keeps its own documentation next to the code: a `docs/wiki/` folder of generated markdown pages, runbooks, conventions. `ExposeDocs` imports those documents into a Beacon project at startup so MCP agents can `search` them and read them with `get_documentation`, alongside the host's data ([Host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/)). It works with `UseAI = false`: no LLM and no embedder is needed. ## Quick start ```csharp var beacon = builder.Services.AddBeaconServices(builder.Configuration, o => { /* your options */ }); beacon.ExposeDbContext(o => o .ReadOnlyConnection("BeaconReadOnly") .AllowTables("Loan", "LoanInstallment", "Customer")); beacon.ExposeDocs(o => { o.ProjectName = "HostContext"; // the project ExposeDbContext uses (its Name/ProjectName) o.FromDirectory("docs/wiki") // relative to ContentRootPath; default glob **/*.md .Exclude("drafts/**"); }); beacon.UseSqlServer(); // ends the builder chain: call ExposeDocs first var app = builder.Build(); // ... apply migrations ... await app.Services.SyncBeaconHostAsync(); // syncs ExposeDbContext data sources, then ExposeDocs documents ``` `SyncBeaconHostAsync` is the single startup entrypoint for everything a host declares. It runs the `ExposeDbContext` sync first (exactly what `SyncBeaconHostDataSourcesAsync` does; that call still works on its own) and then imports the documents. It is a plain startup call, not a hosted service. Both calls are **safe to run from several replicas at once**: the whole sync holds one lock in Beacon's own database (PostgreSQL `pg_try_advisory_lock` on a fixed key, SQL Server `sp_getapplock` with a session owner), so replicas starting together run it one after another instead of racing each other's inserts. A replica waits up to 5 minutes for the lock. A unique-key race that still happens (a row created outside the lock) re-runs the idempotent sync once instead of failing startup. Other providers run the sync unlocked and log a warning. A host that coordinates startup another way can register its own `IHostSyncLock`. ### The host project `ExposeDbContext`, `ExposeDocs` and `AddHostEndpointTools` all resolve their `ProjectName` through one shared resolver, so they always land on the same project. The host project is identified by a **host key** (`Project.HostManagedKey = host:{ProjectName, trimmed and lower-cased}`), never by its display name, which is neither unique nor protected: - On first sync the host project is created with the key. If a project that is **not** host-managed already uses the name (for example one a Beacon user created by hand), it is left untouched and the host project is created as `"{ProjectName} (host)"` (then `"(host 2)"`, …), with a warning in the log. A user project with the same name never receives the host database, documents or endpoint tools. - Renaming the host project in the UI is fine: it is still found by its key. - An archived host project is restored, because the host still declares it. - Changing `ProjectName` creates (or claims) the project for the new name, and the host data source is **unlinked from every other project**, including the project of the old name, so it loses access to the host database. ## Shipping the documents The docs are not part of the build output by default, so the host has to ship them. Either copy them next to the app: ```xml ``` ```csharp o.FromDirectory("docs/wiki"); ``` `FromDirectory` resolves a relative path against `IHostEnvironment.ContentRootPath`. Make sure that is the output directory in your deployment (it is when the app is started from its own folder), or pass an absolute path such as `Path.Combine(AppContext.BaseDirectory, "docs/wiki")`. Or embed them in the assembly: ```xml ``` ```csharp o.FromEmbeddedResources(typeof(Program).Assembly, "docs/wiki/"); ``` The part of the resource name after the prefix is the document path (backslashes become `/`). Without a `LogicalName`, MSBuild mangles folder names (`40-web` becomes `_40_web`) and joins them with dots, so set one. ## Options | Option | Meaning | |---|---| | `ProjectName` | **Required.** The host project the documents attach to, found by its host key (see [The host project](#the-host-project)); created when missing. | | `FromDirectory(path, params globs)` | Files under `path` (relative to the content root, or absolute) matching the globs. Default glob `**/*.md`. | | `FromEmbeddedResources(assembly, prefix)` | Manifest resources whose name starts with `prefix`. | | `Exclude(params globs)` | Skips documents whose relative path matches (applies to every source of this call). | | `MaxDocumentBytes` | Per-document cap, default 512 KB. Larger files are skipped with a warning that names only the path. | `ExposeDocs` may be called more than once, for several folders or for several projects. Only `.md`, `.markdown` and `.txt` files are ingested, whatever the glob matches. Globs must be relative and may not contain `..`; a file that resolves outside the configured directory, and any symbolic link inside it (file or folder), is rejected. A configured directory that does not exist fails startup, so a broken deployment cannot silently archive every imported document. ## What the sync does Each document is stored as a `ProjectImportedDocument`, keyed by project, source (`docs:dir:docs/wiki`, `docs:res:{Assembly}:{prefix}`) and relative path. - **Title**: the frontmatter `title`, else the first `# ` heading (outside code fences), else the file name. - **Frontmatter**: a leading `---` YAML block is stripped from the body and kept as JSON (flat keys, quoted strings, `[a, b]` and `- item` lists, such as a `sources:` list). - **Idempotent**: a document whose content hash is unchanged is left alone; nothing is written when nothing changed. A changed document is updated in place, a removed file is archived, and a file that comes back is un-archived. - **Case-insensitive paths**: source and path are matched ignoring letter case, as SQL Server's unique index compares them. Renaming `Guide.md` to `guide.md` updates the existing row (it takes the new casing). Two files whose paths differ only by case (possible on a case-sensitive file system or in embedded resources) cannot both be stored: the first is imported and the other is skipped with a warning that names its path. - **Chunked for search**: new and changed documents are split into sentence-window chunks (the same chunker and `DocChunkWindowSentences` / `DocChunkOverlapSentences` settings as generated documentation). Keyword search works on them immediately. When the local embedder and semantic retrieval are enabled, the sync then embeds the new chunks through the doc-chunk re-index, and the recurring `mcp-docchunk-reindex` job keeps them current. An embedding failure is logged and does not fail startup. The documents belong to one project and are only ever visible to callers authorized for that project. ## On the MCP surface - **`get_documentation`**: the project-level response (no arguments) ends with an *Imported documents* listing (title + path). `document: ""` (or an exact title) returns one document in full; a title shared by several documents is an error that names their paths. - **`search`**: imported documents match on title, path or chunk text (any word of the query; the documents holding the most words first) and appear as `- **[IMPORTED_DOC]** \`path\` (title)`, followed by ` -- snippet` when there is one, with or without embeddings. - **`get_context` with `format: "agents_md"`**: a deterministic *project brief* in markdown to drop into an agent workspace's `AGENTS.md`: the project description, data sources (host-managed ones marked read-only), up to 30 key tables (best-described and most-related first), masked columns, up to 30 glossary terms, the imported document titles, and how to use Beacon's tools (`get_query_context` → `dry_run` → `query`, read-only, masked columns stay masked; a project without data sources is pointed at `search` and `get_documentation` instead). Built from the catalog only, with no LLM call, and audited like every tool call. The brief opens with a notice that catalog text is untrusted data, prints the project description as one truncated code-span line, and renders every other catalog-derived field (names, identifiers, document paths, glossary text, tool descriptions) as data: on one line, with hidden characters shown as `�`, identifiers and paths in code spans no backtick can close, and Markdown delimiters escaped. It states the read-only and PII-detection guarantees from the project's effective MCP settings (it says so when either is switched off); the masked columns it lists are a host-managed source's host-policy masked columns, which stay masked whatever the PII detection setting. The REST API exposes the same documents: `GET /beacon/api/projects/{id}/imported-documents` (listing) and `GET /beacon/api/projects/{id}/imported-documents/{documentId}` (one document with its content). Both need Read in the project under [project access](https://moberghr.github.io/beacon/security/authorization/#project-access), for sessions and API keys alike; a project the caller cannot read answers `404`. ## Example: data and docs in one project ```csharp beacon.ExposeDbContext(o => { o.Name = "BackOffice"; o.ProjectName = "Back Office"; o.ReadOnlyConnection("BeaconReadOnly").AllowTables("Loan", "LoanInstallment", "Merchant", "Customer"); }); beacon.ExposeDocs(o => { o.ProjectName = "Back Office"; o.FromDirectory("docs/wiki"); }); ``` With the `Content Include` item above, the pages land in `docs/wiki/` next to the host's assembly. On each start only the pages that changed since the last deployment are re-chunked. --- # Knowledge Base & SQL Grounding > How Beacon grounds generated SQL: M-Schema with sample values, schema relationships, a business glossary, human-verified golden examples, learned patterns, and semantic retrieval. Source: https://moberghr.github.io/beacon/mcp/knowledge-base/ Beacon does not pass a bare schema dump to an LLM. Every natural-language question (through the [`ask` tool](https://moberghr.github.io/beacon/mcp/server/#ask), the [MCP Playground](https://moberghr.github.io/beacon/mcp/server/#mcp-playground), or the [`get_query_context` tool](https://moberghr.github.io/beacon/mcp/server/#get_query_context)) is answered against an assembled **grounding context**: the tables that matter, the join paths that are real, the business terms your organisation actually uses, and query examples a human has verified. This page describes what goes into that context, where each piece comes from, and how you curate it. ## The grounding context For a given question and data source, Beacon assembles the context in blocks. Sections marked **(authoritative)** are human-verified; everything else is machine-derived and labelled as such. | Block | Contents | Source | |---|---|---| | **Relevant Tables (full schema)** | M-Schema rendering: column name, type, nullability, description, plus **real sample values** per column | Schema metadata + sample-value collection | | **Other Tables** | Names and primary keys only, so the model knows what else exists without burning the budget | Schema metadata | | **Join Paths (verified)** | Foreign keys and human-confirmed relationships | [Schema relationships](#schema-relationships) | | **Join Paths (UNVERIFIED)** | Relationships pending curator verification, explicitly flagged as needing confirmation: an inferred one with its confidence, one a user added as *added, pending curator verification* | Relationship discovery, people who may not curate the source | | **Coverage** | A note when the table neighbourhood was capped, so the model knows the picture is partial | Context assembly | | **Business Glossary** | Top-K matching terms with definitions and their column/metric mapping | [Business glossary](#business-glossary) | | **Verified query examples (authoritative)** | Human-verified question → SQL pairs for this data source | [Golden examples](#golden-examples-and-the-eval-harness) | | **Learned Patterns (from usage)** | Known corrections, column clarifications, join patterns, common queries, business-term mappings, documentation gaps | [The learning loop](#the-learning-loop) | Sample values are what make filters land on real data (`'shipped'` vs `'SHIPPED'`). They are row data sent to the LLM provider, so collection is off by default: turn `EnableSampleValueCollection` on for a project (its override on **MCP Settings**, `/beacon/mcp-settings`) or globally. A refresh collects while it is on in the effective settings of every live project that links the data source (each project's override, else the global value), so a project that turns it on collects even while the global value is off. On a data source several live projects link, any of them switching it off stops the collection and keeps the stored values out of every project's context (see [Guardrails of shared data sources](https://moberghr.github.io/beacon/security/authorization/#guardrails-of-shared-data-sources)), and the deployment lock `Beacon:Mcp:ForbidRowDataToLlm` keeps it off everywhere. Documentation generation discovers column values for the same projects only, and logs once per run which data sources it listed no values for because collection is off. PII, secret-like and financial-amount columns are never sampled, and a column with a PII- or credential-shaped value keeps none. See [Sample values](https://moberghr.github.io/beacon/features/data-sources/#sample-values) and [What reaches the LLM provider](https://moberghr.github.io/beacon/mcp/server/#what-reaches-the-llm-provider). :::tip Everything on this page is reachable from the outside: `get_query_context` returns the same assembled context an `ask` call would use, so an agent can write its own SQL against the same grounding, validate it with `dry_run`, and execute it with `query`. ::: ## Who curates grounding What a person approves here ends up in every caller's prompts, so curating it is a narrower right than writing: | Grounding | Who curates it | |---|---| | A project's learned patterns (approve, reject, set back to pending), its documentation (editing a section, regenerating it) and documentation patches to a section | **Curate** in the project: a project admin member of it, or an Admin | | A data source's shared grounding: verifying or un-verifying a relationship, deleting a verified one, applying or rejecting a documentation patch to a column or table description | Write where the caller acts **and** Curate in **every** live project linking the source, or the DataSourceAdmin permission (with Write where it acts), or an Admin; a source no live project links: a DataSourceAdmin or an Admin | | Golden examples (promote, edit, deactivate), the [grounding recheck](#rechecking-stored-grounding) and the business glossary | Admins only | Curate is never granted by an open project, and never to an API key (whatever its scope), an MCP caller (user or system, Read or Execute scope) or the owner of scheduled work, whatever their role. Beacon's own background jobs run without a caller and no project check applies to them, but none of them curates: the learning job stores new patterns as `NeedsEvidence`, auto-approves only through the [replay gate](#from-signal-to-injected-lesson) and proposes documentation patches without applying them. A curation that records who decided (reviewing a learned pattern, promoting or editing a golden case, verifying a relationship) also needs a signed-in Beacon user (`403` otherwise). A host that wants only Admins to curate simply adds no project admins; Beacon [warns at startup](#rechecking-stored-grounding) about live projects that learn from feedback while no enabled user is their project admin. The pattern, patch and relationship lists and the project documentation tell each caller per row whether they may curate it (`canCurate`), decided the same way as the routes (`403` otherwise); the UI shows the actions only then, and says who may otherwise. See [Authorization → Project roles](https://moberghr.github.io/beacon/security/authorization/#project-roles). **Grounding SQL is checked and stored without comments.** Every piece of SQL that grounds generation (gold SQL that is promoted or edited, corrected SQL from feedback, a learned pattern's example SQL when it is mined, approved or auto-approved) passes the grounding SQL check (`IGroundingSqlCheck`) before it is stored. The SQL is read in its data source's dialect, its comments are removed, and the comment-free SQL must pass the read-only gate and the catalog check a saved query step passes (a single read-only statement, tables and columns Beacon's metadata holds). Nothing runs. The check refuses, with the reason: - **A comment that cannot be removed safely**: an executable comment (`/*! … */` on MySQL), a comment that is never closed, SQL the dialect's tokenizer cannot read. Comment characters inside a string literal are text, not a comment, and are kept when the dialect's tokenizer reads the literal as the database does: `'a--b'` and `'/* … */'`, `$$…$$` on PostgreSQL, `N'…'` on SQL Server, `'#1'` on MySQL, `'a//b'` on Snowflake, `"a#b"` on BigQuery. Where the engine reads a literal differently from the tokenizer, the SQL is refused, as with `'a--b'` on MySQL, which reads `--` without a following space differently. Comment characters in a quoted identifier (`"a--b"` on PostgreSQL) are refused, and so is any comment marker on a data source whose SQL dialect Beacon does not know. - **Characters a reviewer cannot see**, anywhere in the SQL, a literal or a comment included: format characters (bidirectional overrides and isolates, zero-width characters, the byte order mark), the tag characters U+E0000–U+E007F and variation selectors. - **A data source Beacon holds no tables or columns for**: there is nothing to check the SQL against, so it is refused until metadata is extracted ("Extract metadata for this data source first"). - **SQL the read-only gate or the catalog check refuses**: anything but one read-only statement, or a column the data source's metadata does not hold. A golden case's question holding characters a reviewer cannot see is refused where it is promoted or edited, and a mined example question holding them is not stored. Prompts render gold and example SQL without comments too, including SQL stored before this check existed, and questions on one line. A golden case whose SQL cannot be shown that way, or whose SQL or question holds characters a reviewer cannot see, is left out whole; a pattern loses only its example; a glossary term loses only its metric expression. Beacon logs which ids it left out, by kind, once per hour per row. The documentation export (behind knowledge answers, `get_documentation` and the documentation chunk index) and the prompt that generates documentation show a pattern's example SQL the same way, or not at all. The golden and pattern lists say which rows a prompt leaves out (`goldSqlWithheld`, `questionWithheld`; `exampleSqlWithheld`, `exampleQuestionWithheld`), and the MCP Learning page flags them and shows hidden characters escaped. ### Rechecking stored grounding Grounding stored before these checks existed is not rewritten: it keeps grounding prompts, rendered as above, until an Admin decides about it. `GET /beacon/api/eval/grounding/recheck?projectId={id}` (Admin by live roles, like the rest of `/eval`) runs today's checks over one project's grounding and changes nothing: the question and gold SQL of each active golden case, and the example question and example SQL of each approved or auto-approved learned pattern that is not superseded. SQL passes the grounding SQL check in its data source's dialect (each data source and its catalog are read once per recheck); a golden case's question must not be empty, and no question may hold characters a reviewer cannot see. The answer counts what was checked (`goldenCasesChecked`, `patternsChecked`) and lists every row a check refuses (`refused`, golden cases first): its `kind` (`GoldenCase` or `LearnedPattern`), `id`, `dataSourceId`, the stored `question` and `sql`, the reasons (`questionRefusal`, `sqlRefusal`), a golden case's `promotedByUserId` (null when it was promoted before Beacon recorded who promoted it) and a pattern's `patternStatus` and `contentHash`. A project that does not exist answers `404`. The recheck logs counts and ids only. On the **MCP Learning** page (`/beacon/mcp-learning`) an Admin's **Grounding recheck** tab runs it for a project and lists the refused rows, each with the action the Admin chooses: **Deactivate** a golden case (`PUT /beacon/api/eval/golden/{id}` with `isActive: false`) or set a pattern **Back to pending** (`PUT /beacon/api/mcp/learned-patterns/{id}/status` with `Pending` and the row's `contentHash`). Nothing is deactivated or set back automatically. **Startup warnings.** After the host starts, Beacon warns (best effort: a database it cannot read is noted, never a failure; counts and project ids only) about grounding that waits for an Admin: - the active golden cases that record no promoting user (promoted before Beacon recorded who promoted a case), with their projects: they keep grounding SQL generation, so review them on the **Golden examples** tab, run the recheck, and deactivate the ones nobody stands behind; - with Beacon user management on and learning on (`EnableLearning`, globally and in the project's effective settings), the live projects with no enabled ProjectAdmin member: only an Admin reviews their learned patterns and documentation patches until a member is made their project admin. ## Schema relationships Join paths are the single biggest source of wrong SQL, so Beacon keeps them as first-class, curatable data rather than inferring them at generation time. Each registered relationship maps a source column to a target column and carries an **origin** and a **cardinality**: | Origin | Meaning | Verified by default | |---|---|---| | **Foreign key** | Read from the database's declared constraints | Yes | | **Inferred** | Proposed from column-naming heuristics (`customer_id` → `customers.id`) with a confidence score | No | | **Manual** | Declared by a person in the UI or via the API | Only when its creator [curates the source](#who-curates-grounding) | Cardinality is one of `Unknown`, `One to one`, `One to many`, `Many to many`. Only **verified** relationships are rendered under *Join Paths (verified)*. Unverified relationships appear in a separate block, *pending curator verification*, that tells the model to confirm before relying on them. A wrong join path that looks authoritative is worse than no join path at all. A verified relationship is a join every project linking the data source is told to trust, so only a curator of the source's shared grounding verifies one or deletes one. A relationship someone else declares (or accepts from discovery) is stored **unverified**, pending verification by a curator (the create answer says so with `isVerified: false`). A verified relationship that someone else relabels or gives another cardinality goes back to unverified (the update answer says so the same way). A curator's own declarations and edits stay verified, and a curator's edit records the curator as the relationship's verifier, now. Unverified relationships stay in the prompt, under the unverified block: an inferred one with its confidence, one a user added as *added, pending curator verification*. Every change applies to the relationship as it was read. An edit writes the new label and cardinality together with the verification it leaves, in one statement, and only while the relationship still holds the label, cardinality and verification read; a removal is conditional the same way; a verification names the label and cardinality the curator reviewed (`reviewedLabel`, `reviewedCardinality`) and is written only while the relationship holds them. Anything that lands in between (a curator verifying while an Editor's edit or removal is under way, an Editor's edit while a curator verifies) refuses the later request with `409` ("The relationship changed since you loaded it"), so a relationship is never left verified with content its verifier did not see, and a relationship verified meanwhile is never removed on the strength of a check made while it was unverified. Reload the relationships and try again. ### Managing relationships in the UI Open a data source and go to **Schema relationships** (`/beacon/data-sources/{id}/relationships`): - **Discover** previews inferred relationships without saving anything. Review each proposal with its confidence score and **Accept** the ones that are right. - **Add relationship** declares one by hand. It is verified when you curate the data source; otherwise the page says it is pending verification by a curator, and so does an edit that sends a verified relationship back to unverified. - Existing entries can be verified, un-verified (curators only: for anyone else the action is disabled, with the reason), relabelled or deleted. A verified entry is deleted by curators only; for anyone else the action is disabled, with the reason. The page opens with a **schema health** panel: table count, relationship count, how many are still unverified, the number of connected groups, the largest connected group, isolated tables, and detected junction tables. Isolated tables and a high group count are the signal that the model has no way to join its way to those tables. ### REST API The relationship endpoints sit under the standard authenticated `BeaconApi` policy. They manage grounding metadata that every user's questions depend on, so they are not an admin-only surface. Creating and relabelling a relationship need Write in every live project linking the data source, deleting one Manage there (a DataSourceAdmin: where it acts); verifying one, and deleting a verified one, need [Curate](#who-curates-grounding) there (`403` otherwise). The list returns `canCurate` per relationship; the create answer (`id`, `isVerified`) and the update answer (`200`, `isVerified`) say whether the relationship is verified. The verify body is `isVerified` with, to verify, the `reviewedLabel` and `reviewedCardinality` the curator reviewed (`400` without them); an edit, a removal or a verification of a relationship that changed since it was read answers `409`. | Method | Endpoint | |---|---| | `GET` | `/beacon/api/data-sources/{dataSourceId}/relationships` (filter by `origin`, `verifiedOnly`) | | `POST` | `/beacon/api/data-sources/{dataSourceId}/relationships` | | `PUT` | `/beacon/api/data-sources/{dataSourceId}/relationships/{relationshipId}` | | `POST` | `/beacon/api/data-sources/{dataSourceId}/relationships/{relationshipId}/verify` | | `DELETE` | `/beacon/api/data-sources/{dataSourceId}/relationships/{relationshipId}` | | `POST` | `/beacon/api/data-sources/{dataSourceId}/relationships/discover-preview` | | `GET` | `/beacon/api/data-sources/{dataSourceId}/schema-health` | ## Business glossary The glossary teaches Beacon your organisation's vocabulary: what "active customer", "churn", or "net revenue" means in *your* schema. Terms are project-scoped and optionally narrowed to a single data source. | Field | Purpose | |---|---| | `Term` | The business term as people say it | | `Synonyms` | Alternative phrasings that should match the same term | | `Definition` | Plain-language definition shown to the model | | `TargetSchema` / `TargetTable` / `TargetColumn` | Where the term lives in the schema | | `MetricExpression` | The SQL expression that computes it, when the term is a metric | | `IsActive` | Retire a term without deleting its history | When a question is asked, the top matching terms (`GlossaryTopK`, default 5) are injected into the **Business Glossary** block. Glossary retrieval is scoped to the caller's authorised project: two projects sharing a data source never see each other's definitions. The glossary is managed through the API only and is gated with the Admin policy; there is no UI page for it. | Method | Endpoint | |---|---| | `GET` | `/beacon/api/glossary?projectId={id}&dataSourceId={id}&includeInactive={bool}` | | `POST` | `/beacon/api/glossary` | | `PUT` | `/beacon/api/glossary/{id}` | | `DELETE` | `/beacon/api/glossary/{id}` | ## Golden examples and the eval harness A **golden case** is a human-verified question → SQL pair. Golden cases do double duty: they are injected into the grounding context as *authoritative* examples, and they are the regression suite the eval harness scores generation against. ### How cases are created Only an Admin creates or edits a golden case. Feedback never does: rating an `ask` answer (through the `feedback` MCP tool, the MCP Playground, or `POST /beacon/api/eval/feedback`, which is Admins only) records the verdict, the user who recorded it and any checked corrected SQL on the answer's signal, and nothing else. The verdict is recorded whatever becomes of the corrected SQL: when the project does not retain query content, when the answer did not run on a single data source (a cross-source or knowledge answer: there is no dialect to check the SQL in), or when the [grounding SQL check](#who-curates-grounding) refuses it, only the SQL is dropped, and the answer says so (`correctedSqlRecorded: false` with `correctedSqlNotRecordedReason`). - **Golden candidates.** `GET /beacon/api/eval/golden/candidates?projectId={id}` lists the answers rated `correct` that were not promoted yet. Only `ask` signals count, because a `query` or `dry_run` signal's "question" is the SQL it ran. The list is sorted newest first, and it is paged (`page`, `pageSize`, `sort`). Each candidate shows the question, the SQL a promotion would start from (the rater's corrected SQL, else the signal's own corrected SQL, else its generated SQL) with its `sqlHash` (the SHA-256 of that SQL, lowercase hex), the note, who asked and who rated it. Only signals of one project and one data source whose content was retained are listed. A project under the [content lock](https://moberghr.github.io/beacon/mcp/server/#content-retention) keeps no question to promote. - **By promotion.** `POST /beacon/api/eval/golden/promote` takes the candidate's `signalId`, its `sqlHash` as `reviewedSqlHash`, and optional `notes`, and records the promoting user (`promotedBy`). A promotion applies to what the Admin reviewed: the signal must still be rated `correct` and still hold the SQL whose hash was sent, or the promotion is refused (`409`, "The answer changed since it was reviewed": reload the candidates and review it again). Only an answer to a question becomes a case (`400` for a `query` or `dry_run` signal); a missing signal, one without SQL, project or data source, a data source its project does not link, a question holding characters a reviewer cannot see, and SQL the check refuses answer `400` too, and nothing is created. The promotion claims its signal (`promotedAt`, `promotedEvalCaseId`) in the transaction that creates the case, only while the signal still holds the reviewed SQL (compared by the digest stored with the signal's SQL and written with every change to it, which is exact under any database collation; a signal recorded before Beacon kept the digest gets it first, only while it holds none and the SQL read). A signal is therefore promoted once: promoting it again, or two Admins promoting it at the same moment, answers with the case already promoted from it (`alreadyPromoted: true`, active or not) and creates nothing; a signal promoted before Beacon recorded promotions answers with the first case promoted from it. - **By hand.** `PUT /beacon/api/eval/golden/{id}` edits the question, the gold SQL, the notes, or deactivates the case, and records the editing user (`modifiedBy`). A question holding characters a reviewer cannot see, or gold SQL the check refuses, is refused (`400`) and nothing changes. Promoting and editing need a signed-in Beacon user (`403` otherwise): who did it is recorded. On the **MCP Learning** page an Admin sees three more tabs: **Golden candidates** (filtered by project, paged, each candidate in full with **Promote**; a candidate promoted meanwhile shows its existing case, and one that changed since it was listed says so; reload and review it again), **Golden examples** (a project's cases, showing who promoted and who last edited each, or *Not recorded* for a case created before Beacon recorded it; which cases a prompt leaves out; and **Deactivate**) and **Grounding recheck** (see [Rechecking stored grounding](#rechecking-stored-grounding)). Gold SQL, promoted or edited, and corrected SQL from feedback are stored as the grounding SQL check returns them: without their comments. `GET /beacon/api/eval/golden` returns `promotedByUserId` and `modifiedByUserId` (`Users.Id`) with `promotedByName` and `modifiedByName` (the display name, or "User {id}"); all are empty for a case created before this was recorded. It also returns `goldSqlWithheld` and `questionWithheld`: whether a prompt leaves the case out. ### Injection The top `GoldenExemplarTopK` (default 4) cases matching the question are rendered into **Verified query examples (authoritative)**, ranked *above* mined learned patterns and capped by `GoldenExemplarBudgetChars` (default 2000). Retrieval hard-filters on the caller's project, so golden SQL never leaks across project boundaries on a shared data source: each golden case's vector carries its project, and the nearest-neighbour search takes the top cases of the caller's project only, so another project's cases on the same data source never take its slots. Vectors indexed before they carried a project are stamped with their case's project by the `BackfillGoldenCaseEmbeddingProjects` database migration, without re-embedding them, so golden examples keep matching after the upgrade. Gold SQL is rendered without its comments and the question on one line; a case whose comments cannot be removed, or whose SQL or question holds characters a reviewer cannot see, is left out, and its id is logged once per hour. ### Eval runs `POST /beacon/api/eval/runs` replays every active golden case for a project through the real generation pipeline and scores the result. Comparison is by **result-set fingerprint**. The generated SQL does not have to match the gold text; it has to produce the same rows. Each failure is tagged so that a user can distinguish a retrieval failure (the model never saw the right table) from a SQL-reasoning failure from an execution failure. With `EnableEvalJudge` turned on, an opt-in LLM judge reviews a capped, PII-redacted sample of the rows to distinguish a cosmetically different result from a wrong one. It is off by default. Both the gold SQL and the generated SQL are re-validated read-only and run through the query guardrail before execution. The harness inherits nothing and introduces no new execution path. | Method | Endpoint | |---|---| | `POST` | `/beacon/api/eval/runs` | | `GET` | `/beacon/api/eval/runs?projectId={id}&take={n}` | | `GET` | `/beacon/api/eval/runs/{runId}/results` | | `GET` | `/beacon/api/eval/golden?projectId={id}&dataSourceId={id}` | | `GET` | `/beacon/api/eval/golden/candidates?projectId={id}` | | `GET` | `/beacon/api/eval/grounding/recheck?projectId={id}` | | `PUT` | `/beacon/api/eval/golden/{id}` | | `POST` | `/beacon/api/eval/golden/promote` | | `POST` | `/beacon/api/eval/feedback` | The eval group is Admin-gated: it executes SQL against live data sources and writes golden cases. ## The learning loop Every `ask`, `query`, and `dry_run` call records a **usage signal** holding the question, the generated SQL, the tables and columns referenced, the routing decision, the outcome and timings. A recurring job turns those signals into reusable knowledge. ![MCP Learning](https://moberghr.github.io/beacon/img/screenshots/mcp-learning-dark.png) ### From signal to injected lesson 1. **Mining.** Signals are clustered per project and data source. Only `ask` and `query` signals are mined: `dry_run` rows carry SQL rather than a natural-language question and are excluded so they cannot skew NL pattern mining or documentation-gap error rates. They remain in the signals table for analytics. A pattern's example question comes only from an `ask` signal (a `query` signal's "question" is the SQL it ran), on one line, and is left out when it holds characters a reviewer cannot see. Verdicts weigh in as follows: an `incorrect` verdict keeps the answer out of the common-query and join patterns; a `correct` verdict makes the answer the preferred example of its pattern and raises the pattern's confidence only when a [curator](#who-curates-grounding) recorded it. Where a verdict came from is recorded apart from who it is attributed to: the signal keeps the rating user (`FeedbackByUserId`) and, separately, whether the verdict came from a curator (`FeedbackFromCurator`), true only when an interactive session holding Curate in the signal's project at that moment recorded it (a project admin member or an Admin, signed in). A verdict from an API key or an MCP caller is attributed to the user it acts for and never weighs as a curator's, whatever that user's roles; neither does an Editor's or the asker's own, nor a verdict recorded before Beacon recorded its provenance. Mining reads the recorded provenance, not the rater's current roles. 2. **Lesson extraction.** For each failure-then-correction cluster, an LLM is asked to diagnose the failure and emit one compact, durable lesson. Only the cluster's text fields are sent, never result rows. If the provider is unavailable or the response does not parse, a deterministic regex + column-similarity path produces the lesson instead, so the loop never silently stops. 3. **Classification.** Lessons become learned patterns of type `SchemaCorrection`, `ColumnClarification`, `JoinPattern`, `CommonQuery`, `BusinessTermMapping`, or `DocumentationGap`. 4. **Review.** A new pattern lands as `NeedsEvidence`; confidence alone never approves anything. On the **MCP Learning** page (`/beacon/mcp-learning`) a [curator](#who-curates-grounding) of the pattern's project approves, rejects or sets it back to pending. These are the only statuses a person sets (`PUT /beacon/api/mcp/learned-patterns/{id}/status` answers `400` for any other; `AutoApproved` and `NeedsEvidence` are the job's). A decision applies to the content the reviewer saw: the list hands out each pattern's `contentHash` (the SHA-256 of its content, example question and example SQL), the status change sends it back as `reviewedContentHash`, and a pattern whose content changed since (also when the job refreshes it between the decision's read and its write) is not decided (`409`: reload the patterns and review it again). The same digest is stored with the pattern and written with every change to its content, and the decision's conditional write compares the stored digest. Fixed-length lowercase hex compares exactly under any database collation, where text equality on a case-insensitive SQL Server would let a concurrent case-only or trailing-space change pass. A pattern stored before Beacon kept the digest gets it on its first decision or auto-approval, stored only while the pattern holds no digest and the content read. The reviewer recorded is the caller (`403` for a caller without a Beacon user). The page shows each pattern in full with who reviewed it (*Unknown reviewer* for a review time without a recorded reviewer) and which examples a prompt leaves out; anyone else sees the reason instead of the actions. A pattern's example SQL passes the [grounding SQL check](#who-curates-grounding) when it is mined (SQL that fails is not stored: a new pattern gets none, an existing one keeps its example, and the run logs how many) and again when a person approves the pattern: approval stores it without comments, or is refused with the reason (`400`). Approving a pattern without example SQL checks nothing. 5. **Replay verification.** Auto-approval is decided by replay alone. With `EnableReplayVerification` on (the default), a `NeedsEvidence` candidate is *replayed*: Beacon re-runs the project's relevant golden cases with and without the candidate injected and auto-approves it only if it flips at least `LearningReplayMinFlips` (default 1) case from fail to pass with no regression. Replay is scoped to the candidate's own project and data source. Before it, the candidate's example SQL passes the grounding SQL check again and is replayed without its comments; a candidate whose SQL fails is not replayed and stays `NeedsEvidence`, and the run logs how many. The auto-approval is written only while the candidate is still `NeedsEvidence` and still holds the content the replay measured, compared by its stored digest as a person's decision is. With replay off, nothing is auto-approved: candidates wait for a person. 6. **Injection.** Approved and auto-approved patterns are ranked and injected into **Learned Patterns (from usage)** under `LearningInjectionBudgetChars` (default 1500), filtered to the caller's authorised project. Example SQL is rendered without its comments, read in the pattern's data source's dialect, and the example question on one line; an example that cannot be shown that way is left out and the pattern's content kept. ### Approval binds to the reviewed content The aggregation refresh never rewrites the content, example question or example SQL of a pattern a person approved or rejected. Only its signal count and confidence refresh, so what was reviewed is what is injected. An auto-approved pattern whose content changes goes back to `NeedsEvidence` and is replayed again; the run logs how many went back. Content is compared exactly but for the statistics the job writes into its own sentences (the count of a `CommonQuery` ("… (12 similar queries)") or a `JoinPattern` ("… (8 queries)"), the error rate and query count of a `DocumentationGap` ("… a 40% error rate across 40 queries")), which the pattern's signal count and confidence also hold; any other content, a lesson's numbers included, is compared as written. The question is compared on one line, and the example SQL exactly but for its comments and the whitespace between its tokens, read by the data source's dialect: a string literal, a quoted identifier, a number and an identifier's case compare as written. So a count that moves, or SQL stored with comments before they were removed, is no change: the stored count refreshes and the pattern stays auto-approved, while a changed number in a lesson, a changed literal or an identifier in another case sends it back for replay. Every write the job makes to an existing pattern is conditional on the status it read, so a person's decision between the job's read and its write is never overwritten. ### Documentation patches The learning job does not generate **documentation patches** today: no scheduled job turns approved schema corrections and documentation gaps into a proposed column or table description (or a section of the project's documentation). A patch that is already stored is still reviewed on the **Doc patches** tab, and applying or rejecting one is curation: a section patch needs Curate in its project; a description patch changes the data source's shared metadata, so it needs Curate in every live project linking the source, or a DataSourceAdmin (see [Who curates grounding](#who-curates-grounding)). This is deliberate: a description patch is mined from one project's signals, but the description it writes is what every project linking the source sees, so a project admin of the mining project alone does not apply it. The **Doc patches** tab shows each patch's current and proposed content in full and who applied or rejected it. Patches are scoped project by project: a project's patches came only from its own approved patterns, only for the data sources it linked, and only its own proposed or applied patches stopped a duplicate, so a data source shared by two projects never carries one project's patterns into the other project's patches. This describes patches already stored; no scheduled job generates new ones. A description patch records the description it was proposed on and applies only while the column or table still holds exactly that text. A patch whose target changed since it was proposed, edited by hand or by another project's applied patch on a shared data source, is refused (`409`) and stays proposed; nothing is written and no event is recorded. Reject it; no scheduled job proposes a replacement on the current description. Section patches append to the project's own documentation and are not checked this way. ### Decay A schema correction points at a specific column. When a schema refresh shows that column no longer exists, the correction is marked superseded and is never injected again. The row is kept for audit rather than deleted. ### Retention Signals older than `LearningSignalRetentionDays` (default 90) are removed by the nightly cleanup job. Each project's signals use that project's effective window, and signals recorded without a project use the global one. The cleanup runs whether learning is on or off, so signals recorded while a project's own settings turn learning on still expire when the global switch is off. The window is at least one day: a save with a lower value, globally or for a project, is rejected with `400` (a project's empty value inherits the global window). A window below one day stored before this check is never read as "delete every signal": a project's falls back to the global window, and the global one to the default 90 days. When one project's signals cannot be cleaned up, the job logs the project id and the failure's type, still cleans up every other project and the signals without a project, and then fails, so the run is recorded as failed rather than successful. ## Semantic retrieval Keyword matching alone misses the question that says "revenue" when the column is called `net_amount`. Beacon adds a dense retrieval arm and fuses the two. ### Local embeddings Embeddings are produced **in-process by a local ONNX model**: a 384-dimension bge-small-en-v1.5 with a WordPiece tokenizer and 512-token inputs. There is no network egress and no third-party embedding API. The model and tokenizer are local files you supply. The ONNX support is a separate package, `Moberg.Beacon.AI.Embeddings.Onnx`: add it and call `AddBeaconOnnxEmbeddings()` on the service collection. Embeddings are **disabled by default** so an install without the package or a model keeps working on the lexical path. ```json { "Beacon": { "Embeddings": { "Enabled": true, "ModelPath": "/opt/beacon/models/bge-small-en-v1.5.onnx", "TokenizerPath": "/opt/beacon/models/vocab.txt" } } } ``` On PostgreSQL, vectors are additionally written to a **pgvector** column with an HNSW cosine index so nearest-neighbour search runs in the database. This requires the pgvector extension to be available on the server, since the migration that adds the column runs `CREATE EXTENSION IF NOT EXISTS vector`. On SQL Server the same vectors are compared in memory from their byte representation, with no extension required. Semantic retrieval works on both; PostgreSQL is just faster at it. ### Fusion Where a dense arm is available, its ranked results are merged with the keyword results using **reciprocal rank fusion**, with deterministic tie-breaks so paging stays stable. This drives: - The `search` MCP tool across tables, columns, and documentation. - Exemplar selection for golden examples and learned patterns (`ExemplarTopK`, default 4). A learned pattern is indexed only while its project links the data source and is not archived; its vector, like a golden case's, carries its own project, and selection filters on the caller's project, so a shared data source never surfaces another project's patterns or cases. - Documentation chunk retrieval (`DocChunkTopK`, default 6). Every dense path **fails open**: with no embedder configured, or with rows not yet indexed, retrieval falls back to the keyword/overlap path and returns results rather than failing. ### Documentation chunking Generated project documentation is split into overlapping sentence windows (`DocChunkWindowSentences`, default 5; `DocChunkOverlapSentences`, default 1) and embedded so a question can retrieve the paragraph that answers it rather than a whole document. With `EnableContextualRetrieval` enabled, each chunk is additionally given a short LLM-written blurb situating it in its parent document before embedding, which gives more accurate retrieval at the cost of one LLM call per chunk at index time. Fusion for documentation is at **section** granularity: two chunks of the same section collapse into one result. ## Background jobs Indexing and aggregation run as recurring [Warp](https://moberghr.github.io/warp/) jobs in the host, visible on the `/warp` dashboard: | Job | Schedule | What it does | |---|---|---| | `mcp-learning-aggregate` | Every 6 hours | Mines signals into learned patterns, runs decay and replay verification | | `mcp-learning-cleanup` | Daily at 03:00 | Removes signals past the retention window | | `mcp-embedding-reindex` | Every 12 hours | Re-embeds schema metadata and exemplars | | `mcp-docchunk-reindex` | Every 12 hours | Re-chunks and re-embeds project documentation, each project under its own effective settings (a project that turned semantic or contextual retrieval off is skipped or gets no blurbs) | | `mcp-audit-retention` | Daily at 03:30 | Purges MCP audit rows older than `Beacon:Mcp:Audit:RetentionDays` (a no-op while it is unset); see [MCP observability](https://moberghr.github.io/beacon/mcp/observability-and-audit/#retention) | These are the sample host's registrations (`Beacon.SampleProject/Program.cs`); a host that embeds Beacon schedules them itself. ## Settings reference These are the grounding, retrieval and learning settings of `McpSettingsData`; they round-trip through `GET`/`PUT /beacon/api/mcp/settings`. The guardrail settings (max row limit, read-only enforcement, PII detection and patterns, content retention) are described in [MCP Server → Configuration](https://moberghr.github.io/beacon/mcp/server/#configuration). The **MCP Settings** page (`/beacon/mcp-settings`) shows sample-value collection and value grounding (globally and per project) and the learning switch, auto-approve threshold, injection budget and signal retention. Everything else in this table is set through the API only; a save on the page sends every setting back as it read it (the stored values, `stored` on the `GET`), so the API-only ones keep their values. | Setting | Default | Effect | |---|---|---| | `EnableSampleValueCollection` | `false` | Collect real sample values for the M-Schema block (and discover column values for documentation) | | `EnableValueGrounding` | `false` | Probe live values for the literal-looking words of a question and add the matches to the context | | `ValueGroundingMaxProbes` | `12` | Most value probes per question; on a shared data source the lowest value among the linking projects wins. Each probe has 5 seconds from its start (a slower probe matches nothing; the others keep their matches), and all the probes of one question stop at 15 seconds. They run one at a time, or up to three at a time while the caller's credential has `MaxConcurrentQueriesPerKey` slots free: each probe beside the call counts as one more query of that credential | | `EnableLearning` | `true` | Master switch for signal mining | | `LearningAutoApproveThreshold` | `0.7` | Stored with the settings; promotion is decided by replay verification (`EnableReplayVerification`, `LearningReplayMinFlips`) | | `LearningInjectionBudgetChars` | `1500` | Character budget for the learned-patterns block | | `LearningSignalRetentionDays` | `90` | Signal retention window, in days; at least 1 | | `EnableReplayVerification` | `true` | Auto-approve a `NeedsEvidence` candidate only when golden-case replay shows it helps; off, nothing is auto-approved | | `LearningReplayMinFlips` | `1` | Fail→pass flips a candidate must produce to be promoted | | `EnableSemanticRetrieval` | `true` | Use the dense arm when an embedder is available | | `ExemplarTopK` | `4` | Exemplars selected per question | | `EnableGoldenExemplars` | `true` | Inject human-verified examples | | `GoldenExemplarTopK` | `4` | Golden cases injected | | `GoldenExemplarBudgetChars` | `2000` | Character budget for the golden-examples block | | `GlossaryTopK` | `5` | Glossary terms injected | | `EnableContextualRetrieval` | `false` | LLM-written situating blurb per doc chunk at index time | | `DocChunkWindowSentences` | `5` | Sentences per documentation chunk | | `DocChunkOverlapSentences` | `1` | Sentence overlap between chunks | | `DocChunkTopK` | `6` | Documentation chunks retrieved | | `EnableSelfConsistency` | `false` | Generate several SQL candidates and vote | | `SelfConsistencyCandidateCount` | `5` | Candidates generated when voting is on | | `SelfConsistencyMinTables` | `2` | Voting runs only for questions whose context spans at least this many tables | | `EnableSemanticLint` | `true` | Deterministic checks on generated SQL in `ask` (an undeclared join, an aggregate fanning out over a one-to-many join, a non-aggregated column missing from `GROUP BY`), with one repair attempt from the shared repair budget | | `EnableEvalJudge` | `false` | LLM judge adjudicates near-miss eval results | ## Project isolation Grounding is retrieval, and retrieval is not authorisation, so every retrieval path is explicitly scoped to the project the caller is authorised for, not to the data source. On a data source shared by several projects, the glossary terms, golden examples, learned patterns, and replay cases one project sees are only ever its own. The same holds for every grounding call that names a data source, and for every host that calls Beacon's AI services directly: each takes the caller's authorized project (`projectId`) and reads the data source only through it. - **Project material** (DataModel documentation, code references from the project's repositories, glossary terms, golden examples and learned patterns) comes from that project only, never from another project that links the same data source. - **A data source the project does not link** is not searched, described or catalogued: the table and column search finds nothing in it, the LLM context, table knowledge, lineage, data source overview and table schemas for it answer "not found", its schema catalog is empty (so every table in SQL against it fails the schema check), and value grounding probes nothing. - **Live value probes need Query.** Value grounding probes the live data only for a caller who may query the project; a caller who may only read it (`get_query_context` as a Viewer) is grounded with the stored sample values alone. - **The stored catalog is cached per project and data source.** `ask`, `get_query_context`, `query` and `dry_run` read a data source's stored tables and columns from a snapshot kept for up to 15 minutes, one per project. The project's link is checked on every call before the snapshot is read, and the stored sample values are screened under the project's settings on every call. A metadata refresh, a host model sync, an applied description patch, and a data source update, enable, disable, delete or migration-write mark drop the source's snapshots at once on the instance that made the change; another instance serves its own snapshot until it expires, including sample values that the change cleared (an archive, a change of target or a migration-write mark). The data source itself is refused on every instance at once when it is archived, disabled or marked as a write target. - **Code references** belong to the repository that holds them, and a repository to one project: a scan names its project. - An eval run needs a project (`400` without one). ## See Also - [MCP Server](https://moberghr.github.io/beacon/mcp/server/): the tools that consume this context, and the guardrails around them - [AI Integration](https://moberghr.github.io/beacon/features/ai-integration/): documentation generation and the LLM provider abstraction - [Data Sources](https://moberghr.github.io/beacon/features/data-sources/): schema metadata loading - [Admin Settings](https://moberghr.github.io/beacon/security/admin-settings/): LLM provider configuration --- # Integrate Beacon into your application > The step-by-step checklist for embedding Beacon in an ASP.NET Core host: packages, Program.cs, database, sign-in, jobs, connectors, AI, MCP and production, written for people and for coding agents. Source: https://moberghr.github.io/beacon/integrate/ Beacon ships as a set of NuGet packages (`Moberg.Beacon.*`). You add them to an ASP.NET Core application, wire them in `Program.cs`, and your application serves Beacon's UI at `/beacon`, its REST API at `/beacon/api`, and its MCP server at `/beacon/mcp`, under your own sign-in, your own job runner and your own deployment. This section is the complete integration guide. This page is the checklist; each step links to a page with every option for that step. :::tip[Using Claude Code or another coding agent?] Point it at the whole guide in one file (for an MCP gateway of your own application, use `llms-gateway.txt` instead): ```text wrap Integrate Beacon into this ASP.NET Core app. Read https://moberghr.github.io/beacon/llms-integrate.txt first and follow its checklist step by step. Ask me before every decision the guide marks as one. ``` Every page of these docs is also available as Markdown: add `.md` to the page URL, or use **Copy as Markdown** under the page title. ::: ## Which integration? | You want | Follow | |---|---| | An MCP server for **your own application**: AI clients act as your signed-in users through your own endpoints and read your docs. No data sources, no SQL. | [MCP gateway for your application](https://moberghr.github.io/beacon/mcp/gateway/), one page, `llms-gateway.txt` | | **Beacon's data product** in your application: monitored databases, queries, alerts, data movement, and an MCP server over them. | This checklist, or the shorter recipe [Embed Beacon (PostgreSQL + Warp)](https://moberghr.github.io/beacon/cookbooks/embed-postgresql-warp/) | The two combine (a gateway host can add data sources later), but start with the one you need: the gateway path leaves out most of this checklist (scheduler, connectors, login form, first-run setup) on purpose. ## Before you start | You need | Notes | |---|---| | .NET 10 SDK | Every Beacon package targets `net10.0`. | | An ASP.NET Core host | A minimal-API or MVC app. Beacon adds endpoints and middleware to it. | | A database for Beacon's own data | PostgreSQL with the `pgvector` extension available, or SQL Server. Beacon keeps everything in the `beacon` schema. See [Metadata database](https://moberghr.github.io/beacon/integrate/database/). | | A job runner | Beacon does not run background work itself. You implement `IBeaconScheduler` on your scheduler of choice; [Moberg Warp](https://moberghr.github.io/warp/) is the reference. See [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/). | | An encryption key | 32 random bytes, base64-encoded, from a secret store. Beacon refuses to start without one. | | Node.js | Only if you build the UI package from source. Consumers of `Moberg.Beacon.UI` get the built SPA in the package. | ## Decisions to make first These choices shape the wiring. An agent should ask, not guess. | Decision | Options | Default if you have no preference | |---|---|---| | Metadata database | PostgreSQL (`Moberg.Beacon.Core.PostgreSql`) or SQL Server (`Moberg.Beacon.Core.SqlServer`) | PostgreSQL | | Who signs users in | Beacon's own user store and login form, your OIDC provider (Entra ID or any OIDC), or your app's existing sign-in | Beacon user store + OIDC if you already have an identity provider | | Job runner | Warp, Hangfire, Quartz.NET, or your own | Warp | | Which engines Beacon monitors | Any of nine connectors | Only the engines you have | | AI features | On (needs an LLM provider: OpenAI, Anthropic, Azure OpenAI or AWS Bedrock) or off | Off until you have a provider; MCP still needs the AI package | | MCP server | Exposed to AI clients or not | Exposed, with API keys | | Change approvals | Four-eyes review of query, migration, Custom SQL and recipient changes, off by default | On for regulated deployments | | Data migration (the only write path) | Off (default) or on | Off | ## The checklist 1. **Add the packages.** At minimum `Moberg.Beacon.Core`, a metadata-database provider, `Moberg.Beacon.Api`, `Moberg.Beacon.UI`, and one connector per engine you monitor. Add `Moberg.Beacon.AI` + `Moberg.Beacon.MCP` for the MCP server. → [Packages](https://moberghr.github.io/beacon/integrate/packages/) 2. **Set the two required secrets.** `ConnectionStrings:BeaconContext` and `Beacon:EncryptionKey` (`openssl rand -base64 32`). Never commit either. → [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/#configuration) 3. **Implement `IBeaconScheduler`** on your job runner, and make sure every job marks its DI scope as system work. Without a scheduler `AddBeaconServices` throws at startup. → [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/) 4. **Register services in `Program.cs`:** `AddBeaconServices(...)` with your options, connectors, then `UsePostgreSql()` / `UseSqlServer()` last in the chain; cookie authentication with base path `/beacon`; Data Protection; `AddBeaconApiAuthorization()`, `AddBeaconApiServices()`, antiforgery; `AddBeaconAI` and `AddBeaconMcp` if you use them. → [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/) 5. **Build the middleware pipeline in the documented order.** The order is load-bearing: MCP challenge → API keys → JWT bearer → authentication → Beacon cookie → authorization → login form → first-run setup → antiforgery. → [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/#the-middleware-pipeline) 6. **Map the endpoints:** `MapBeaconApi()`, `MapLoginEndpoints("/beacon", …)`, `MapSetupEndpoints("/beacon")`, `MapBeaconMcp()`, `MapMcpDiscovery()`, `MapBeaconUi()`. → [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/#endpoints) 7. **Migrate Beacon's schema at startup** with `Beacon.Core.ServiceConfiguration.UseBeacon(app.Services, createSchema: true)`. Beacon does not migrate itself. → [Metadata database](https://moberghr.github.io/beacon/integrate/database/) 8. **Choose how people sign in:** Beacon's login form, OIDC/SSO (tenant allow-list, guest blocking, MFA), bearer JWTs, API keys. Plan the first-run admin and a break-glass account. → [Sign-in & identity](https://moberghr.github.io/beacon/integrate/authentication/) 9. **Decide who sees what:** projects are the access boundary; functional permissions split the Admin role; optional four-eyes approvals. → [Access control](https://moberghr.github.io/beacon/integrate/access-control/) 10. **Configure connectors** for the engines you monitor, each with a read-only database login. → [Data-source connectors](https://moberghr.github.io/beacon/integrate/connectors/) 11. **Optional, AI (required for MCP):** set `UseAI = true`, call `AddBeaconAI`, configure the LLM provider and, optionally, local embeddings. → [AI providers](https://moberghr.github.io/beacon/integrate/ai/) 12. **Optional, MCP (needs step 11):** expose `/beacon/mcp`, set `Beacon:PublicBaseUrl`, issue API keys or map Entra callers, connect Claude, Cursor or VS Code. → [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/) (in the [MCP](https://moberghr.github.io/beacon/mcp/) section) 13. **Optional, extend:** expose your own EF Core `DbContext`, your Markdown docs, or your own HTTP endpoints as MCP tools. → [Extending Beacon](https://moberghr.github.io/beacon/integrate/extending/) 14. **Harden for production:** Data Protection keys encrypted at rest, forwarded headers behind a proxy, deployment locks, audit retention, setup token, session affinity for MCP. → [Production checklist](https://moberghr.github.io/beacon/integrate/production/) 15. **Verify.** Start the app and check, in this order: - The console prints `Beacon first-run setup token (enter it on the setup page): …` (user management on, empty database). - `/beacon/setup` creates the first administrator, in the browser, or [headless](https://moberghr.github.io/beacon/integrate/authentication/#headless-first-run-scripts-and-ci). - `GET /beacon/api/health` answers `200`, also before the first admin exists (`503` means Beacon cannot reach its database). - You can sign in and see **Home** at `/beacon/home`. - `GET /.well-known/mcp/server-card.json` lists the eight MCP tools (if you mapped MCP). - An API key with the Execute scope can list the MCP tools. See [Check it works](https://moberghr.github.io/beacon/mcp/connect/#check-it-works). If something fails, see [Troubleshooting](https://moberghr.github.io/beacon/integrate/troubleshooting/). ## What you get ```mermaid flowchart LR subgraph Host["Your ASP.NET Core host"] direction TB UI["/beacon: React UI
Moberg.Beacon.UI"] API["/beacon/api: REST + SignalR hub
Moberg.Beacon.Api"] MCP["/beacon/mcp: MCP server
Moberg.Beacon.MCP"] Core["Beacon.Core: handlers, project access,
SQL execution gate, notifications"] Jobs["Your job runner
implements IBeaconScheduler"] UI --> API --> Core MCP --> Core Jobs --> Core end Core --> Meta[("Beacon metadata DB
schema beacon")] Core --> Sources[("Your databases
read-only logins")] Core --> Out["Email · Teams · Slack · Jira · Webhook"] Core --> LLM["LLM provider (optional)"] ``` Beacon never takes over the root of your site: everything lives under `/beacon` and `/.well-known/…` (MCP discovery). Your own routes, auth schemes and pages keep working. ## The reference host `src/Beacon.SampleProject/Program.cs` in the repository is a complete, running composition with every connector, Warp jobs, OIDC, bearer JWT for MCP, OpenTelemetry and forwarded headers. It is a sample, never shipped as a package. [Run it](https://moberghr.github.io/beacon/getting-started/quick-start/) to see a working Beacon before you embed one, and compare your `Program.cs` with it when something behaves differently. --- # Packages > Every Moberg.Beacon NuGet package, what it contains, which ones you need, and how versions are published. Source: https://moberghr.github.io/beacon/integrate/packages/ Beacon is published to nuget.org as `Moberg.Beacon.*` packages. Package IDs carry the `Moberg.` prefix; assemblies and namespaces stay `Beacon.*` (`using Beacon.Core;`). Every package targets `net10.0`. ## Which packages you need | Package | Contains | Needed when | |---|---|---| | `Moberg.Beacon.Core` | Domain, MediatR handlers, EF model, project access, SQL execution gate, notifications, scheduling contracts | Always (every other package depends on it) | | `Moberg.Beacon.Core.PostgreSql` | Beacon's metadata database on PostgreSQL: `UsePostgreSql(...)`, migrations, pgvector support | Beacon's own data lives in PostgreSQL | | `Moberg.Beacon.Core.SqlServer` | Beacon's metadata database on SQL Server: `UseSqlServer(...)`, migrations | Beacon's own data lives in SQL Server | | `Moberg.Beacon.Api` | REST endpoints under `/beacon/api`, the SignalR hub, cookie/OIDC/JWT/API-key authentication, login and first-run setup endpoints, antiforgery, Data Protection helpers | You serve the UI or the REST API (in practice always) | | `Moberg.Beacon.UI` | The built React app as a Razor Class Library, served at `/beacon` | You want the UI | | `Moberg.Beacon.AI` | LLM providers (OpenAI, Azure OpenAI and Claude over plain HTTP, no vendor SDK), the request queue, documentation generation, grounding, learning loop, AI actors | AI features, and always with MCP (MCP depends on it) | | `Moberg.Beacon.AI.Bedrock` | The Amazon Bedrock LLM provider (`AddBeaconBedrock()`) | You use Bedrock as the LLM provider | | `Moberg.Beacon.AI.Embeddings.Onnx` | Local in-process ONNX embeddings (`AddBeaconOnnxEmbeddings()`) | You want semantic retrieval; without it retrieval stays keyword-based | | `Moberg.Beacon.MCP` | The MCP server at `/beacon/mcp`, its tools, filters, discovery documents, host endpoint tools | You expose Beacon to AI clients | | `Moberg.Beacon.Gateway` | `AddBeaconMcpGateway`: Beacon as the MCP gateway of your own application — its users as callers, its endpoints as tools, its documents as grounding, no data sources | You want an MCP server for your application rather than Beacon's data product | | `Moberg.Beacon.Testing` | Test helpers for a gateway host: Entra-shaped test tokens, an in-memory user resolver, an MCP client over your test server | You test a gateway host's MCP tools | | `Moberg.Beacon.Doctor` | The `beacon-doctor` .NET tool: checks a deployed MCP endpoint the way an MCP client sees it | You deploy an MCP endpoint (a tool, not a library reference) | | `Moberg.Beacon.Connector.PostgreSql` | PostgreSQL connector, and the PostgreSQL migration writer | You monitor PostgreSQL, or migrate data into it | | `Moberg.Beacon.Connector.SqlServer` | SQL Server connector, and the SQL Server migration writer | You monitor SQL Server, or migrate data into it | | `Moberg.Beacon.Connector.MySql` | MySQL connector, and the MySQL migration writer | You monitor MySQL, or migrate data into it | | `Moberg.Beacon.Connector.AzureSynapse` | Azure Synapse connector | You monitor Synapse | | `Moberg.Beacon.Connector.Snowflake` | Snowflake connector | You monitor Snowflake | | `Moberg.Beacon.Connector.BigQuery` | Google BigQuery connector | You monitor BigQuery | | `Moberg.Beacon.Connector.Databricks` | Databricks SQL connector | You monitor Databricks | | `Moberg.Beacon.Connector.CloudWatch` | AWS CloudWatch Logs Insights connector | You monitor CloudWatch | | `Moberg.Beacon.Connector.Api` | REST API connector (OpenAPI import, JSON to rows) | You monitor HTTP APIs | :::caution[Two different "SQL Server" packages] `Moberg.Beacon.Core.SqlServer` stores **Beacon's own data** in SQL Server (`UseSqlServer`). `Moberg.Beacon.Connector.SqlServer` lets Beacon **monitor** a SQL Server database (`AddSqlServerConnector`). If both are true, add both. The same split applies to PostgreSQL. ::: ## Typical sets **PostgreSQL metadata, monitoring PostgreSQL, with UI and MCP:** ```bash dotnet add package Moberg.Beacon.Core dotnet add package Moberg.Beacon.Core.PostgreSql dotnet add package Moberg.Beacon.Api dotnet add package Moberg.Beacon.UI dotnet add package Moberg.Beacon.AI dotnet add package Moberg.Beacon.MCP dotnet add package Moberg.Beacon.Connector.PostgreSql ``` **SQL Server shop, no AI, no MCP:** ```bash dotnet add package Moberg.Beacon.Core dotnet add package Moberg.Beacon.Core.SqlServer dotnet add package Moberg.Beacon.Api dotnet add package Moberg.Beacon.UI dotnet add package Moberg.Beacon.Connector.SqlServer ``` Add your job runner too. For the reference implementation, use the `Moberg.Warp.*` packages (see [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/)). ## Dependency direction ```text Moberg.Beacon.Api → Core Moberg.Beacon.AI → Core Moberg.Beacon.AI.Bedrock → AI Moberg.Beacon.AI.Embeddings.Onnx → AI Moberg.Beacon.MCP → Core, AI Moberg.Beacon.Gateway → Core, Api, AI, MCP, UI Moberg.Beacon.Testing → Gateway Moberg.Beacon.Core.* → Core Moberg.Beacon.Connector.* → Core Moberg.Beacon.UI → (nothing; static web assets) ``` `Moberg.Beacon.Core` itself references no server database driver or EF Core provider (it carries `Microsoft.Data.Sqlite` for the in-memory joins of cross-source queries). The SQL Server EF provider comes only with `Moberg.Beacon.Core.SqlServer`; Npgsql and pgvector only with `Moberg.Beacon.Core.PostgreSql` or the PostgreSQL connector; each connector brings its own driver. Data-migration writes go through the destination's connector (`IMigrationBulkWriter`, registered by `AddConnector()`), so Core carries no bulk-copy library either. For scale: a PostgreSQL host with Core, Core.PostgreSql, Api, UI, AI, MCP and Connector.PostgreSql restores 45 third-party packages. The ONNX runtime's native binaries and the AWS SDK arrive only with their optional packages. ## Versions - There is no version in the source code. The git release tag is the package version: publishing a GitHub release packs every packable project with that tag and pushes it to nuget.org. Four-part tags are normalised by NuGet (a tag `4.5.0.0` publishes `4.5.0`). - Keep every `Moberg.Beacon.*` package on the **same version**. They are built and released together. - These docs track the `main` branch. Check the [release notes](https://github.com/moberghr/beacon/releases) of the version you install for what it includes. If you use central package management, pin one version for all of them: ```xml ``` ## Build-time notes - **`Microsoft.AspNetCore.OpenApi`.** If your host calls `AddOpenApi()`/`MapOpenApi()`, reference `Microsoft.AspNetCore.OpenApi` directly; relying on the copy that comes with `Moberg.Beacon.Api` fails the build with `CS9137` (interceptors). - **Package downgrade (`NU1109`).** Under central package management, restore fails with "Detected package downgrade" when your `Directory.Packages.props` pins a package lower than a Beacon package needs (for example EF Core or `Microsoft.Extensions.*` one patch behind, or an older `ClosedXML`). Raise each pin to the version the message names. - **Transitive package advisories.** If restore reports `NU1902`/`NU1903` for a transitive dependency and you treat warnings as errors, add a direct `PackageReference` to the patched version of that package in your project. - **The UI package** contains the production React build. You do not need Node.js to consume it. The host must call `app.UseStaticFiles()` so the assets are served. - **Building from source** (project references instead of packages): `Beacon.UI` builds the React app only in `Release`, or in `Debug` with `-p:BuildReactInDebug=true`. Node.js 22 and npm 11 match the CI build. ## Licence Beacon is dual-licensed: GNU AGPL v3.0, or a commercial licence without copyleft obligations. Moberg partners use it under a licence exception. Embedding Beacon in a network service under the AGPL triggers its source-disclosure terms (§13). See [LICENSING.md](https://github.com/moberghr/beacon/blob/main/LICENSING.md). --- # Host wiring > A complete, annotated Program.cs for embedding Beacon: service registration, the load-bearing middleware order, endpoint mapping, startup migration, and every option of AddBeaconServices. Source: https://moberghr.github.io/beacon/integrate/host-setup/ This page gives you a complete `Program.cs` and explains each part. Copy it, then remove what you do not use. The reference host in the repository, `src/Beacon.SampleProject/Program.cs`, follows the same shape. ## Configuration Two values are required before anything starts. Keep both out of source control: user-secrets in development, a secret store or environment variables everywhere else. | Key | Environment variable | Value | |---|---|---| | `ConnectionStrings:BeaconContext` | `ConnectionStrings__BeaconContext` | Connection string of Beacon's metadata database | | `Beacon:EncryptionKey` | `Beacon__EncryptionKey` | Base64 of at least 32 random bytes: `openssl rand -base64 32` | ```bash dotnet user-secrets init dotnet user-secrets set "ConnectionStrings:BeaconContext" "Host=localhost;Database=beacon;Username=beacon;Password=…" dotnet user-secrets set "Beacon:EncryptionKey" "$(openssl rand -base64 32)" ``` The database must exist; Beacon creates its schema and tables in it (see [Metadata database](https://moberghr.github.io/beacon/integrate/database/)). Beacon checks the encryption key in every environment and refuses to start when it is missing, shorter than 32 bytes, not random-looking (all printable ASCII, or fewer than 16 distinct byte values), a published example value, or an unexpanded `${…}` placeholder. Outside `Development` you also need `Beacon:DataProtection:ApplicationName` (see [Data Protection](#data-protection)). The [configuration reference](https://moberghr.github.io/beacon/getting-started/configuration/) lists every other key. ## The complete Program.cs This host keeps Beacon's metadata in PostgreSQL, monitors PostgreSQL, signs users in with Beacon's own user store (plus optional OIDC), runs jobs on Warp, and exposes the MCP server. `BeaconScheduler` and `BeaconClaimsTransformation` are small classes you add to your project. See [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/) and [the claims transformation](#the-claims-transformation) below. ```csharp using Beacon.AI; // AddBeaconAI using Beacon.Api; // AddBeaconApiServices using Beacon.Api.Authentication; // cookie/OIDC/JWT/Data Protection helpers + middleware using Beacon.Api.Endpoints; // AddBeaconApiAuthorization, MapBeaconApi, MapLoginEndpoints, MapSetupEndpoints using Beacon.Connector.PostgreSql; // AddPostgreSqlConnector using Beacon.Core; // AddBeaconServices, BeaconConfiguration using Beacon.Core.Authentication.Providers; // DatabaseAuthenticationProvider using Beacon.Core.PostgreSql; // UsePostgreSql using Beacon.MCP; // AddBeaconMcp, MapBeaconMcp using Beacon.MCP.Discovery; // MapMcpDiscovery, UseMcpResourceMetadataChallenge using Beacon.UI; // MapBeaconUi, BeaconUiEndpointRouteBuilderExtensions.BasePath var builder = WebApplication.CreateBuilder(args); const string beaconBasePath = BeaconUiEndpointRouteBuilderExtensions.BasePath; // "/beacon" // AI work (documentation, ask) can run for minutes: Kestrel's keep-alive and request-header timeouts and the default // HttpClient timeout become 5 minutes (host-wide settings, so Beacon applies them only when you call this). builder.ConfigureBeaconTimeouts(); // Beacon's own routes get body-size and request-time bounds from Beacon:RequestLimits (getting-started/configuration); // the 5-minute values above stay, AI calls need them. A shorter RequestHeadersTimeout (30 s) is fine: headers arrive at once. // Your job runner (here: Warp). See "Background jobs". builder.Services.AddBeaconWarp(builder.Configuration); // Claims transformation (the SignalR user id provider comes with AddBeaconApiServices). builder.Services.AddBeaconHostInfrastructure(); // 1. Beacon core, connectors, metadata database (UsePostgreSql ends the chain). builder.Services.AddBeaconServices(builder.Configuration, options => { options.AddBeaconScheduler(); // required options.BaseUrl = "https://beacon.example.com/beacon"; // deep links in notifications; include /beacon options.UseAI = true; // needs AddBeaconAI below options.Authorization.Enabled = true; options.Authentication.EnableLoginForm = true; options.AddAuthenticationProvider(); options.EnableUserManagement(); // options.EnableApprovalWorkflow(x => x.RequireApprovedVersionToRun = true); // four-eyes, opt-in // options.AddEmailAdapter(); // your IEmailAdapter }) .AddPostgreSqlConnector() .UsePostgreSql(); // ConnectionStrings:BeaconContext // 2. Sign-in. builder.Services.AddBeaconCookieAuthentication(beaconBasePath); builder.Services.AddBeaconDataProtection(builder.Configuration, builder.Environment); // Outside Development chain .ProtectKeysWithCertificate(...) or .ProtectKeysWithAzureKeyVault(...) here. builder.Services.AddBeaconOidcAuthentication(builder.Configuration); // no-op unless Beacon:Authentication:Oidc:Enabled builder.Services.AddBeaconOidcMcpJwtAuthentication(builder.Configuration); // bearer JWTs, only with Oidc:McpJwksEndpoint // 3. AI and MCP (optional). builder.Services.AddBeaconAI(builder.Configuration); builder.Services.AddBeaconMcp(); // 4. REST API, policies, realtime, antiforgery, OpenAPI. builder.Services.AddAntiforgery(x => { x.HeaderName = "X-XSRF-TOKEN"; x.Cookie.Name = ".Beacon.Antiforgery"; }); builder.Services.AddBeaconApiAuthorization(); // BeaconApi, Admin, Execute-scope and permission policies builder.Services.AddBeaconApiServices(); // SignalR hub + notifiers; x => x.Realtime = false to turn off builder.Services.AddBeaconOpenApi(); // the v1 document, with named enum schemas for the generated client var app = builder.Build(); var beaconConfiguration = app.Services.GetRequiredService(); // Beacon does not migrate itself: create the schema and apply migrations at startup. Beacon.Core.ServiceConfiguration.UseBeacon(app.Services, createSchema: true); // ── Middleware: the order below is load-bearing ────────────────────────── app.UseHttpsRedirection(); // see "HTTPS redirection" below for local MCP over HTTP app.UseBeaconSecurityHeaders(); // CSP, nosniff, framing and referrer headers on Beacon's routes app.UseStaticFiles(); // serves the UI bundle app.UseBeaconApiExceptionHandler(); // problem+json for exceptions under /beacon/api app.UseMcpResourceMetadataChallenge(); // adds resource_metadata to 401s from /beacon/mcp app.UseBeaconAuthentication(); // API key, bearer JWT, UseAuthentication, Beacon.Auth cookie, origin check app.UseAuthorization(); app.UseMiddleware(beaconConfiguration, beaconBasePath); app.UseMiddleware(beaconConfiguration, beaconBasePath); app.UseAntiforgery(); // ── Endpoints ──────────────────────────────────────────────────────────── app.MapBeaconOpenApi(); // /openapi/v1.json (signed-in callers only outside Development) app.MapBeaconApi(); // /beacon/api/* and the /beacon/api/hub SignalR hub app.MapLoginEndpoints(beaconBasePath, beaconConfiguration); // /beacon/api/auth/login, logout, sso/challenge app.MapSetupEndpoints(beaconBasePath); // /beacon/api/setup/* (first-run admin) app.MapBeaconMcp(); // /beacon/mcp, Execute scope required app.MapMcpDiscovery(); // /.well-known/oauth-protected-resource, server card app.MapBeaconUi(); // React app at /beacon await app.StartBeaconWarpAsync(); // your job runner's startup (see "Background jobs") app.Run(); ``` `AddBeaconWarp` and `StartBeaconWarpAsync` are two small extension methods you write once; [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/) shows them. With another job runner, replace both lines. `AddBeaconOpenApi()` (in `Beacon.Api.OpenApi`) is `AddOpenApi()` with the settings Beacon's generated TypeScript client needs: enums stay integers on the wire and each one becomes a component schema named after its type, with its values and names. Its optional argument configures the document further. `MapOpenApi()` needs a direct reference to `Microsoft.AspNetCore.OpenApi` in your project (`dotnet add package Microsoft.AspNetCore.OpenApi`); getting it only through `Moberg.Beacon.Api` fails the build with `CS9137`. Drop both lines if you do not want an OpenAPI document. ## What each part does ### `AddBeaconServices` ```csharp public static BeaconBuilder AddBeaconServices( this IServiceCollection services, IConfiguration configuration, Action beaconConfiguration) ``` It registers Beacon's core services and returns a `BeaconBuilder`. Chain connectors and host extensions on the builder, and finish with `UsePostgreSql(...)` or `UseSqlServer(...)`. Those return `IServiceCollection`, so they end the chain. ```csharp builder.Services.AddBeaconServices(builder.Configuration, options => { /* … */ }) .AddPostgreSqlConnector() // connectors: BeaconBuilder → BeaconBuilder .ExposeDbContext(…) // host extensions: BeaconBuilder → BeaconBuilder .UsePostgreSql(); // metadata DB: BeaconBuilder → IServiceCollection (last) ``` `BeaconConfiguration` is set **in code** through the lambda; it is not bound from `appsettings.json`. | Option | Default | What it does | |---|---|---| | `AddBeaconScheduler()` | None (required) | Your `IBeaconScheduler`. Without one, `AddBeaconServices` throws `Implementation of IBeaconScheduler is required.` | | `BaseUrl` | `null` | Public URL of the UI, **including `/beacon`** (e.g. `https://beacon.example.com/beacon`). Notification messages link to `{BaseUrl}/notifications/{id}`. Editable later in **Admin settings**. | | `UseAI` | `false` | Turns AI features on. Also call `AddBeaconAI`. | | `Authorization.Enabled` | `false` | Role-based authorization. Turn it on in every real deployment. | | `AddAuthorizationProvider()` | user store when user management is on, otherwise allow-all | Custom `IBeaconAuthorizationProvider` (global read/write ceiling per user). | | `Authentication.EnableLoginForm` | `false` | Beacon's login page and the redirect of signed-out browsers to `/beacon/login`. | | `AddAuthenticationProvider()` | user store when user management is on | Password check. `DatabaseAuthenticationProvider` uses Beacon's user store. | | `Authentication.CookieExpirationHours` / `RememberMeExpirationDays` | `24` / `30` | Requested cookie lifetime; always capped by the session's absolute lifetime (12 h by default). | | `Authentication.LoginRedirectPath` | `/beacon/home` | Where the login page sends a user after sign-in. | | `EnableUserManagement(x => …)` | off | Beacon's user store: users, roles, first-run setup, password policy (`MinimumPasswordLength` 8, `RequirePasswordComplexity` true, `AllowInternalUsers` true). | | `EnableApprovalWorkflow(x => …)` | off | Four-eyes review of query changes and, by default, migration jobs, Custom SQL rules and recipient destinations; data-source connections and the LLM connection when switched on. See [Approvals](https://moberghr.github.io/beacon/security/approvals/). | | `AddEmailAdapter()` | none | Your `IEmailAdapter`. Without one there is no email channel. | | `SavedQueryTools.NamedToolLimit` | `25` | Above this many published saved queries, MCP lists two generic tools instead of one tool per query. | Only two parts of user management are also read from configuration: `Beacon:UserManagement:SetupToken` and `Beacon:UserManagement:LocalLogin:*`. ### The metadata database `UsePostgreSql()` and `UseSqlServer()` read `ConnectionStrings:BeaconContext`. Overloads take a connection string, or, on PostgreSQL, your own `NpgsqlDataSource`. The schema is always `beacon`. Details: [Metadata database](https://moberghr.github.io/beacon/integrate/database/). ### Authentication helpers | Call | What it does | |---|---| | `AddBeaconHostInfrastructure()` | Registers your claims transformation. | | `AddBeaconCookieAuthentication(basePath)` | The `Beacon.Auth` cookie (HttpOnly, SameSite=Lax, Secure on HTTPS), server-side session revalidation, JSON 401/403 for API and MCP calls instead of redirects. **Pass `"/beacon"`**, so the login path is `/beacon/login`. | | `AddBeaconDataProtection(configuration, environment)` | Stores the Data Protection key ring in Beacon's database. Skip it if your host already configures Data Protection. | | `AddBeaconOidcAuthentication(configuration)` | OIDC sign-in from `Beacon:Authentication:Oidc` (does nothing unless `Enabled`). | | `AddBeaconJwtAuthentication(jwt => …)` | Bearer JWTs for the REST API and MCP (code-configured). | All options are on [Sign-in & identity](https://moberghr.github.io/beacon/integrate/authentication/). ### API services | Call | What it does | |---|---| | `AddBeaconApiAuthorization()` | Registers the `BeaconApi`, `BeaconApiAdmin` and Execute-scope policies plus one policy per functional permission, and the UI's page policies `BeaconUiPage` and `BeaconUiSetupPage`. Required by `MapBeaconApi`, `MapBeaconMcp` and `MapBeaconUi` (which fails at startup without it). | | `AddBeaconApiServices()` | SignalR with Beacon's hub user-id provider, the login and request rate limiters (`Beacon:RateLimits`), notifiers, and the request principal Beacon's core reads the caller from (`IBeaconRequestPrincipal`; `AddBeaconCookieAuthentication` and `AddBeaconMcp` register it too, and startup fails when Beacon's endpoints are mapped without it). Options: `Realtime` (default `true`), `ConfigureSignalR` (e.g. a Redis backplane), `ExternalLogin` (when your app owns sign-in). | | `AddAntiforgery(...)` | The UI sends the antiforgery token in `X-XSRF-TOKEN`; it gets it from `GET /beacon/api/csrf`. API-key and bearer callers are exempt. | :::caution[Upgrading to 5.0: the request principal] In 5.0 an `IHttpContextAccessor` alone no longer tells Beacon's core who the request's caller is. A web host calls `AddBeaconApiServices`, `AddBeaconCookieAuthentication` or `AddBeaconMcp` (each registers the request principal), or registers its own with `UseRequestPrincipal()`. A host that registers only Beacon's core and an accessor serves every request as if no request were in progress. ::: ```csharp // Realtime off: no SignalR, no hub route; the UI simply does not connect. builder.Services.AddBeaconApiServices(x => x.Realtime = false); // Several instances: a SignalR backplane. builder.Services.AddBeaconApiServices(x => x.ConfigureSignalR = signalR => signalR.AddStackExchangeRedis(redisConnectionString)); // Your app owns sign-in: the UI sends signed-out users to your login page. builder.Services.AddBeaconApiServices(x => x.ExternalLogin = new ExternalLoginOptions { LoginUrl = "/account/login", ReturnUrlParameter = "returnUrl" }); ``` The hub addresses a signed-in session by its `NameIdentifier`, the user's `Users.ExternalId` (the `UserId` of `IBeaconActorAccessor`'s actor). Publish your own events with `IHubContext.Clients.User(actor.UserId)`. API keys and MCP callers get no hub user id. Register your own `IUserIdProvider` before `AddBeaconApiServices()` to replace Beacon's. ## The middleware pipeline The order is load-bearing. Each step depends on the ones before it: | # | Middleware | Why it sits here | |---|---|---| | 1 | `UseForwardedHeaders` (behind a proxy) | Everything after it must see the client's address and scheme: the login throttle is per IP, the cookie's Secure flag follows the scheme, the origin check compares the scheme and Host. | | 2 | `UseHttpsRedirection` | API keys travel in headers; never accept them over plain HTTP outside development. | | 3 | `UseBeaconSecurityHeaders` | Security response headers on Beacon's routes, before `UseStaticFiles` so the UI bundle carries them too. `UseBeaconAuthentication` adds them itself when you leave this out. See [Security headers and the origin check](#security-headers-and-the-origin-check). | | 4 | `UseStaticFiles` | Serves the UI bundle before authentication runs. | | 5 | `UseBeaconApiExceptionHandler` (optional) | Turns Beacon's exceptions under `/beacon/api` into problem+json. See [below](#error-responses). | | 6 | `UseMcpResourceMetadataChallenge` | Decorates every 401 from `/beacon/mcp` with the RFC 9728 `resource_metadata` pointer; must precede the middleware that writes those 401s. | | 7 | `UseAuthentication` (`UseBeaconAuthentication` wires steps 7 to 9) | Your own schemes. A remote sign-in callback (the OIDC form post) is answered here, before the origin check. | | 8 | Beacon's authentication | One step: a presented API key (`Authorization: Bearer sk-sem_…`) or bearer JWT (Entra and other OIDC access tokens, when enabled), else the `Beacon.Auth` cookie, fills `HttpContext.User` **before** authorization evaluates policies. A presented key or token wins over any session, so the key's scope and projects apply and an MCP client is never redirected to a login page. | | 9 | Origin check | Checks the Host and `Origin` of state-changing, realtime-hub and anonymous requests (`Beacon:Security`). | | 10 | `UseAuthorization` | Endpoint policies. | | 11 | `LoginFormAuthMiddleware` | Redirects signed-out browsers to `/beacon/login` (only when `EnableLoginForm`) from Beacon routes whose endpoint neither allows anonymous callers nor carries its own authorization. | | 12 | `FirstRunSetupMiddleware` | Until the first admin exists, redirects Beacon's requests to `/beacon/setup`, except the health check `/beacon/api/health` and the MCP discovery documents (only with user management). | | 13 | `UseAntiforgery` | After authentication, so tokens are issued for the signed-in user. | A refused API key is answered with a 401 problem (`application/problem+json`) on every Beacon route, also an anonymous one. A refused bearer token is answered with a 401 problem carrying `error: "invalid_token"` (and, for 5.x clients, `message`) and `WWW-Authenticate: Bearer error="invalid_token"` on an API request; on a page it is ignored, so the session cookie or the login redirect applies. Beacon registers its schemes under the names in `BeaconAuthenticationDefaults` (the `Beacon` policy scheme forwards to the one the request's credential belongs to); they never take away a host's only scheme of its own as its default. Do not call `UseAuthentication()` again after `UseBeaconAuthentication()`: a second pass of your schemes would replace the API key or token Beacon's step accepted. Steps 3, 8, 9, 11 and 12 act on `/beacon` and the prefixes in `Beacon:Authentication:AdditionalPaths` only: a Beacon API key, bearer token or `Beacon.Auth` cookie never authenticates another route of your application, and your routes are never redirected to Beacon's login or setup page. Beacon's cookie scheme refuses the session on those other routes as well, so a host whose default scheme is the cookie still never has its own routes authenticated by it. List `/warp` there when you serve the Warp dashboard with Beacon's admin check: a host upgraded from a version without path scoping must add it, or the dashboard refuses every request. See [Where Beacon credentials apply](https://moberghr.github.io/beacon/getting-started/configuration/#where-beacon-credentials-apply). :::danger[Do not reorder or assign `context.User` later] Beacon's authentication records its result as the request's authentication result; SignalR uses it to close connections when a session ends. Reordering it, or assigning `context.User` after it, breaks API-key callers, the login redirect and session revocation. `ApiKeyAuthMiddleware` and `BeaconCookieAuthMiddleware` are obsolete: call `UseBeaconAuthentication` instead. ::: ## Endpoints | Call | Maps | |---|---| | `MapBeaconApi()` | `/beacon/api/*` (cookie or API key, `BeaconApi` policy, antiforgery and permission filters) and the SignalR hub `/beacon/api/hub`. Migration endpoints only when `Beacon:Migrations:Enabled`. | | `MapLoginEndpoints("/beacon", configuration)` | `POST /beacon/api/auth/login` (only when password sign-in is offered), `POST /beacon/api/auth/logout`, `GET /beacon/api/auth/sso/challenge`. | | `MapSetupEndpoints("/beacon")` | `GET /beacon/api/setup/status`, `POST /beacon/api/setup/superadmin`, `GET /beacon/api/setup/roles`. Extension on `WebApplication`. Map it only with user management on. | | `MapBeaconMcp()` | `/beacon/mcp` with the Execute-scope policy. Never map MCP yourself with `MapMcp(...)` and a bare `RequireAuthorization()`. | | `MapMcpDiscovery()` | Anonymous `/.well-known/oauth-protected-resource`, `/.well-known/oauth-protected-resource/beacon/mcp`, `/.well-known/mcp/server-card.json`. | | `MapBeaconUi()` | The React app at `/beacon` and its client routes. Your app keeps `/`. With the login form on, its pages admit signed-in callers only and send others to the login page, also without `LoginFormAuthMiddleware` (through the cookie scheme's challenge: a host that sets another `DefaultChallengeScheme` gets that scheme's challenge instead); the login, logout and error pages are anonymous, and the setup page is while user management is on. With the login form off, the pages follow your `FallbackPolicy` when you set one, and are open otherwise. | | `MapBeaconOpenApi()` | Your OpenAPI document, which includes Beacon's endpoints. Optional. Same as `MapOpenApi()`, but outside Development it needs a signed-in Beacon caller (list `/openapi` in `Beacon:Authentication:AdditionalPaths`) unless `Beacon:OpenApi:Anonymous` is `true`. A host that calls `MapOpenApi()` itself keeps it as it is. | The full route list is on [Routes & endpoints](https://moberghr.github.io/beacon/reference/routes/). ## The claims transformation `AddBeaconHostInfrastructure()` needs an `IClaimsTransformation`. With Beacon's user store, this one adds the user id and name claims and resolves a global role from the store by the session's external id, never from a name the user typed. It behaves like the reference host's `SampleClaimsTransformation`; Beacon's own role constants are internal, so it carries the role names and levels itself: ```csharp using System.Security.Claims; using Beacon.Core.Authorization; using Beacon.Core.Models.UserManagement; using Beacon.Core.Services; using Microsoft.AspNetCore.Authentication; public sealed class BeaconClaimsTransformation(IUserManagementService users) : IClaimsTransformation { public async Task TransformAsync(ClaimsPrincipal principal) { if (principal.Identity is not ClaimsIdentity identity || !identity.IsAuthenticated) { return principal; } var name = identity.Name ?? "unknown"; if (!identity.HasClaim(x => x.Type == BeaconClaims.UserId)) { identity.AddClaim(new Claim(BeaconClaims.UserId, name)); } if (!identity.HasClaim(x => x.Type == BeaconClaims.UserName)) { identity.AddClaim(new Claim(BeaconClaims.UserName, name)); } var externalId = identity.FindFirst(ClaimTypes.NameIdentifier)?.Value; if (identity.HasClaim(x => x.Type == BeaconClaims.Role) || string.IsNullOrWhiteSpace(externalId)) { return principal; } var user = await users.GetUserByExternalIdAsync(externalId); var role = user is { IsEnabled: true } ? ResolveRole(user) : null; if (role != null) { identity.AddClaim(new Claim(BeaconClaims.Role, role)); } return principal; } private static string? ResolveRole(BeaconUserData user) { if (user.IsSuperAdmin) { return "Admin"; } if (user.Roles.Count == 0) { return null; } // Role levels: Admin 3, Editor 2, Viewer 1. Functional roles (level 0) carry no global role. var level = user.Roles.Max(x => x.Level); return level switch { >= 3 => "Admin", 2 => "Editor", 1 => "Viewer", _ => null, }; } } ``` ## Error responses Beacon's handlers throw `InvalidOperationException` for business-rule violations, `BeaconException` for domain errors and `SettingLockedException` when a deployment lock refuses a change; access refusals are already mapped to 403/404 by `MapBeaconApi`. `app.UseBeaconApiExceptionHandler()` (in `Beacon.Api.Middleware`) maps the rest to RFC 7807 `application/problem+json` for requests under `/beacon/api`, so the UI shows the message instead of a generic 500: | Exception | Status | `type` | `title` | |---|---|---|---| | `BeaconForbiddenException`, `UnauthorizedAccessException` | 403 | `/errors/forbidden` | `Forbidden` | | `InvalidOperationException` | 400 | `/errors/invalid-operation` | its message | | `SettingLockedException` | 409 | `/errors/setting-locked` | its message | | other `BeaconException` | 400 | `/errors/beacon` | its message | | anything else | 500 | `/errors/internal` | `Internal server error` (the exception is logged, never returned) | Other paths of your application are not touched, and neither is a response that has already started or a request the client cancelled. Place it after `UseStaticFiles` and before the authentication middleware, so their exceptions are mapped too. `UseBeaconPipeline()` adds it as its first step. ## Data Protection The `Beacon.Auth` cookie and antiforgery tokens are protected with ASP.NET Core Data Protection, which belongs to your host: - **Your host already configures Data Protection** (shared key ring, Azure Blob, Redis): skip `AddBeaconDataProtection`. - **Otherwise** call `AddBeaconDataProtection(configuration, environment)`. It keeps the key ring in Beacon's database (`data_protection_keys`) under the application name `Beacon:DataProtection:ApplicationName`, which is required outside `Development` and takes one name per environment. Outside `Development` the key ring must be encrypted at rest or the host does not start. Chain a protector: ```csharp builder.Services.AddBeaconDataProtection(builder.Configuration, builder.Environment) .ProtectKeysWithCertificate(certificate); // or .ProtectKeysWithAzureKeyVault(keyId, credential) ``` `Beacon:DataProtection:AllowUnencryptedKeys=true` acknowledges an unencrypted ring instead (logged as a warning). With encryption on, keys that were stored unencrypted are revoked at startup, so everyone signs in once more. ## HTTPS redirection `UseHttpsRedirection` belongs in every deployment: API keys travel in headers. On a development machine running plain HTTP, local MCP clients connect over HTTP, so the reference host skips the redirect for `/beacon/mcp` in Development only: ```csharp if (app.Environment.IsDevelopment()) { app.UseWhen(x => !x.Request.Path.StartsWithSegments("/beacon/mcp"), y => y.UseHttpsRedirection()); } else { app.UseHttpsRedirection(); } ``` ## Security headers and the origin check Beacon protects its own routes in the browser; your routes keep whatever your application sends. Both act on `/beacon` and `Beacon:Authentication:AdditionalPaths` only. **Response headers.** Every Beacon response carries `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy: strict-origin` and a `Permissions-Policy` that turns off camera, microphone, geolocation, payment and similar device features. Responses under `/beacon` also carry a Content-Security-Policy that fits the shipped UI: ```text default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; font-src 'self' data:; connect-src 'self'; worker-src 'self'; object-src 'none'; base-uri 'self'; form-action 'self'; frame-ancestors 'none' ``` Scripts, styles, fonts and workers load only from Beacon's own origin, nothing loads from a CDN, and no page can frame Beacon. `style-src` allows inline styles because the SQL editor and the diagram renderer inject style elements. `Beacon:Security:Csp` is `Enforce` (default), `ReportOnly` (sent as `Content-Security-Policy-Report-Only`, which the browser only reports in its console) or `Off`. A header your application or a proxy already set on the response is kept, so you can send a policy of your own. **Origin check.** It checks the requests a page on another origin could send: `POST`, `PUT`, `PATCH` and `DELETE`, the realtime hub at `/beacon/api/hub` (negotiate and the WebSocket upgrade) and Beacon's anonymous endpoints (sign-in, first-run setup, health). Their `Origin` must be an allowed origin; without an `Origin`, the browser's `Sec-Fetch-Site` must not be `cross-site` or `same-site`. The allowed origins are `Beacon:Security:AllowedOrigins` plus the origin of `Beacon:PublicBaseUrl`; when neither is set, the request's own scheme and Host. With either set, a browser request must also arrive with one of their hosts, so Beacon answers browsers only under the names you configured. Clients that are not browsers (API keys, MCP clients, health probes) send neither header and are not affected, and a request that carries an `Authorization` header (API key, bearer token) is not checked at all: a page on another origin cannot add one without a CORS preflight, so only cookie-authenticated requests need the check. Set `Beacon:Security:AllowedOrigins` or `Beacon:PublicBaseUrl` before you enforce: without them the check compares the `Origin` with the request's own scheme and Host (behind a TLS-terminating proxy that does not forward the scheme, every browser write is refused, and Beacon logs a startup warning in `Enforce`), and the Host check does not run, so there is no DNS-rebinding protection. In the MCP gateway the check runs before routing, so the anonymous-endpoint rule does not apply there; the state-changing and hub rules do. Configured hosts are compared in their punycode form. `Beacon:Security:OriginGuard` is `LogOnly` (default for this release: a request it would refuse goes on and is logged as a warning naming the method, path and reason) or `Enforce` (answered 403 problem+json). Run a release in `LogOnly`, look for the warnings, then switch to `Enforce`. ```json { "Beacon": { "PublicBaseUrl": "https://beacon.example.com", "Security": { "Csp": "Enforce", "OriginGuard": "Enforce", "AllowedOrigins": [ "https://beacon.example.com" ] } } } ``` Behind a reverse proxy, set `Beacon:PublicBaseUrl` (or list the public origin in `AllowedOrigins`), and let the proxy forward the public `Host` or apply forwarded headers: with configured origins, a browser request that arrives with the proxy's internal host name is refused under `Enforce`. An origin is a scheme, a host and an optional port, with no path; an entry that is not fails startup. **HSTS** is yours: `Strict-Transport-Security` binds the whole host name, so send it from your application (`app.UseHsts()` outside Development) or the proxy. Set `AllowedHosts` to your real host names as well, so the framework's host filtering refuses other names on every route. ## The fluent chain Since 5.x the same host can be wired with one chain on the services and two calls on the app. The chain is the preferred way to wire Beacon. The calls above keep working: `AddBeaconServices`, `AddBeaconAI` and `AddBeaconMcp` stay supported and become obsolete in a later release. ```csharp using Beacon.AI; // AddAI using Beacon.Api; // AddApi using Beacon.Api.Authentication; // AddCookieAuth using Beacon.Gateway; // UseBeaconPipeline, MapBeacon (package Moberg.Beacon.Gateway) using Beacon.MCP; // AddMcp builder.Services.AddBeacon(builder.Configuration, options => { options.AddBeaconScheduler(); options.Authentication.EnableLoginForm = true; options.EnableUserManagement(); }) .AddPostgreSqlConnector() .AddAI() // switches AI on; never set UseAI yourself .AddMcp() // after AddAI .AddApi() // REST API, its policies, antiforgery (X-XSRF-TOKEN, .Beacon.Antiforgery) .AddCookieAuth() // Beacon.Auth cookie at /beacon + AddBeaconHostInfrastructure .AddOidc() // optional: SSO, and bearer JWTs for MCP with Oidc:McpJwksEndpoint .UsePostgreSql(); // ends the chain: it returns IServiceCollection var app = builder.Build(); app.UseStaticFiles(); app.UseBeaconPipeline(); // steps 5 to 13 of the pipeline table above, in that order app.MapBeacon(); // API, sign-in, setup, MCP with the Execute policy, discovery documents, UI ``` | Call | Does | |---|---| | `AddBeacon(configuration, options)` | `AddBeaconServices` without `UseAI`: setting `UseAI = true` there throws, so AI is on only when `.AddAI()` registered it. | | `.AddAI()` | Registers Beacon's AI services, sets `UseAI` and removes the AI-off stand-ins Core registered, so the real eval service and connection tester are the ones resolved. A second call throws, and so does a call beside `AddBeaconAI` or `UseAI = true`. | | `.AddMcp(configure?)` | `AddBeaconMcp`. Throws when `.AddAI()` has not run or `AddBeaconMcp` already did. | | `.AddApi(configure?)` | `AddBeaconApiServices`, `AddBeaconApiAuthorization` and antiforgery with the header the shell sends (`X-XSRF-TOKEN`) and the cookie name `.Beacon.Antiforgery`. These are the application's antiforgery options, shared with your own antiforgery. | | `.AddCookieAuth()` | `AddBeaconHostInfrastructure()` and `AddBeaconCookieAuthentication("/beacon")`. | | `.AddOidc()` | `AddBeaconOidcAuthentication` and `AddBeaconOidcMcpJwtAuthentication`: SSO from `Beacon:Authentication:Oidc` and, when it names a `McpJwksEndpoint`, bearer JWTs validated against those keys with the `Authority` as issuer and the `ClientId` as audience. With SSO off it registers only the options. | | `app.UseBeaconPipeline()` | `UseBeaconApiExceptionHandler`, `UseMcpResourceMetadataChallenge` (with MCP), `UseBeaconAuthentication`, `UseAuthorization`, `LoginFormAuthMiddleware` (with `EnableLoginForm`), `FirstRunSetupMiddleware` (with user management), `UseAntiforgery` (with antiforgery registered). Call it once, in place of `UseAuthentication`/`UseAuthorization`; a second call throws. | | `app.MapBeacon()` | With `.AddApi()`: `MapBeaconApi`, `MapLoginEndpoints` (with `.AddCookieAuth()`, since sign-in issues the `Beacon.Auth` cookie), `MapSetupEndpoints` (with user management) and `MapBeaconUi`. With `.AddMcp()`: `MapBeaconMcp` and `MapMcpDiscovery`. `MapOpenApi` stays yours. | `UseForwardedHeaders`, `UseHttpsRedirection`, `UseBeaconSecurityHeaders`, `UseStaticFiles`, `MapOpenApi` (or `MapBeaconOpenApi`), a bearer JWT setup of your own and Data Protection stay host calls, and so do the optional helpers `builder.ConfigureBeaconTimeouts()` and `AddBeaconOpenApi()`. A host adopting `UseBeaconPipeline` gets problem+json errors on `/beacon/api` (see [Error responses](#error-responses)); remove an exception handler of your own for those paths, and keep yours for the rest of your application. Use either the chain or the older calls for AI and MCP, not both: once the chain is in use, `AddBeaconAI` and `AddBeaconMcp` throw, and `.AddAI()` or `.AddMcp()` beside them throw too. A host that embeds Beacon as an MCP gateway keeps `AddBeaconMcpGateway`, `UseBeaconMcpGateway` and `MapBeaconMcpGateway` (the gateway is a profile of the same chain with AI left off, so the evaluation harness keeps refusing); `UseBeaconPipeline` and `MapBeacon` throw on it. ## Minimal variants **No MCP, no AI.** Drop the `Moberg.Beacon.AI` and `Moberg.Beacon.MCP` packages, `UseAI`, `AddBeaconAI`, `AddBeaconMcp`, `UseMcpResourceMetadataChallenge`, `MapBeaconMcp` and `MapMcpDiscovery`. **Your app owns sign-in.** Leave user management and the login form off, set `BeaconApiOptions.ExternalLogin`, and make sure your authentication produces a principal with `ClaimTypes.NameIdentifier` and a Beacon role claim (`Admin`, `Editor` or `Viewer`). Without Beacon's user store, only the `Admin` role claim grants functional permissions unless `Beacon:Authorization:Roles:HonourHostRoleClaims` is `true`. See [Sign-in & identity](https://moberghr.github.io/beacon/integrate/authentication/#your-app-owns-sign-in). **SQL Server metadata.** Replace `Moberg.Beacon.Core.PostgreSql` with `Moberg.Beacon.Core.SqlServer` and `UsePostgreSql()` with `UseSqlServer()`. Warp needs PostgreSQL for its own tables; with SQL Server, use another runner or give Warp a PostgreSQL database. --- # Metadata database > Where Beacon keeps its own data: PostgreSQL with pgvector or SQL Server, the fixed beacon schema, how to create and migrate it, and how to hand Beacon your own NpgsqlDataSource. Source: https://moberghr.github.io/beacon/integrate/database/ Beacon stores its own data (users, projects, data sources, queries, subscriptions, runs, audit rows, grounding) in a **metadata database**. This is separate from the databases Beacon monitors, although it can live on the same server. ## Choose a provider | | PostgreSQL | SQL Server | |---|---|---| | Package | `Moberg.Beacon.Core.PostgreSql` | `Moberg.Beacon.Core.SqlServer` | | Registration | `.UsePostgreSql(...)` | `.UseSqlServer(...)` | | Naming | snake_case tables and columns | Pascal case | | Semantic retrieval | `pgvector` column + HNSW index | Vectors compared in memory | | Extra requirement | The `vector` extension must be available on the server | None | Both providers carry their own EF Core migrations and are kept in step: every schema change ships for both. ## The `beacon` schema Everything lives in the schema `beacon`, including the migration history table (`beacon.__EFMigrationsHistory`). The schema is **not configurable**: migration snapshots name it, so `Beacon:Schema` set to anything other than `beacon` makes startup fail with an explanation. Leave the key unset. ## Registration ```csharp // Connection string from ConnectionStrings:BeaconContext .UsePostgreSql(); // Explicit connection string .UsePostgreSql(connectionString); // Your own NpgsqlDataSource: Aspire, Entra passwordless auth, a shared pool .UsePostgreSql(sp => sp.GetRequiredService()); .UsePostgreSql(dataSource); // SQL Server .UseSqlServer(); .UseSqlServer(connectionString); ``` The overloads that also take a `schema` argument are obsolete (removed in 6.0): they accept only `"beacon"` and otherwise ignore it. `UsePostgreSql` / `UseSqlServer` register an `IDbContextFactory` and a scoped `BeaconContext` (Data Protection persists its key ring through it). Call them **last** in the `AddBeaconServices(...)` chain: they return `IServiceCollection`, and connectors, `ExposeDbContext` and `ExposeDocs` must come before. ### Bringing your own `NpgsqlDataSource` When you pass a data source (or a factory resolved once from the root provider), Beacon: - runs `SET search_path TO "beacon", public` on every connection it opens, so its unqualified SQL and the `vector` type resolve; - does not need `UseVector()` on your data source, because it never binds the vector type; - requires connection reset to stay on (no `No Reset On Close=true`) and `Multiplexing` off, and refuses a data source with either the first time it builds its context: the search path it sets would otherwise survive the connection's return to the pool, or leak into commands sharing it, and run your own queries against Beacon's schema. With a connection string, Beacon builds its own data source with `SearchPath=beacon,public`. ## Creating and migrating the schema **Beacon does not migrate its schema by itself.** Create the **database** first (`CREATE DATABASE beacon;`, or `POSTGRES_DB=beacon` for the container image). Beacon creates the schema and tables, not the database. Then call `UseBeacon` once at startup, after `builder.Build()`: ```csharp var app = builder.Build(); // Creates the beacon schema if missing, then applies pending EF Core migrations. Beacon.Core.ServiceConfiguration.UseBeacon(app.Services, createSchema: true); ``` `UseBeacon` is a plain static method on `Beacon.Core.ServiceConfiguration` (several Beacon packages have a class called `ServiceConfiguration`; use the full name). It is synchronous. | Situation | Approach | |---|---| | One instance, or a dev machine | `UseBeacon(app.Services, createSchema: true)` at startup. | | Several instances starting together | Run migrations once, from a deployment step, a dedicated migrator process, or an idempotent SQL script, rather than from every replica. | A migrator process that does not compose Beacon (it only shares the application's database) calls the standalone migrator of its provider instead of `UseBeacon`, which would need Beacon's whole application layer registered: ```csharp // PostgreSQL, over the application's own data source (an Entra-token one, say) or a connection string. await Beacon.Core.PostgreSql.BeaconPostgreSqlMigrator.MigrateAsync(dataSource); // SQL Server. await Beacon.Core.SqlServer.BeaconSqlServerMigrator.MigrateAsync(connectionString); ``` Both create the `beacon` schema when it is missing and apply the pending migrations; both are idempotent. Over a shared PostgreSQL data source the migrator refuses one with `Multiplexing` or `No Reset On Close`, as `UsePostgreSql` does. | The application login cannot create schemas or extensions | Pre-create the `beacon` schema (and on PostgreSQL the `vector` extension) as an administrator, grant the application login ownership of the schema, then call `UseBeacon(app.Services)` without `createSchema`. | ### pgvector on PostgreSQL One migration runs `CREATE EXTENSION IF NOT EXISTS vector;` and adds a `vector(384)` column with an HNSW index. The extension must be **available** on the server (`SELECT * FROM pg_available_extensions WHERE name = 'vector';` returns a row) and the migrating login must be allowed to create it, or an administrator must install it first. - Azure Database for PostgreSQL – Flexible Server: add `VECTOR` to the `azure.extensions` server parameter. - Amazon RDS / Aurora PostgreSQL: pgvector is available on current engine versions. - Self-hosted: install the pgvector package for your PostgreSQL major version, then restart. If the extension is installed in `public`, Beacon's search path (`beacon, public`) finds it. ### Generating SQL instead of migrating at runtime Migrations live in the provider assemblies. To produce a script from a checkout of the Beacon repository: ```bash dotnet tool install --global dotnet-ef dotnet ef migrations script --idempotent \ --project src/Beacon.Core.PostgreSql \ --startup-project src/Beacon.SampleProject \ --context PostgreSqlBeaconContext \ --output beacon-postgresql.sql ``` The startup project needs `ConnectionStrings:BeaconContext` configured. `--context` is required because the sample host has more than one `DbContext`. The migrations create the `beacon` schema and the `vector` extension themselves. ## Sizing and housekeeping - Run history, MCP audit rows and learning signals grow with use. Set `Beacon:Mcp:Audit:RetentionDays` and schedule the audit purge (see [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/#recurring-maintenance-jobs)); learning signals are cleaned by the `CleanupOldSignals` job after the project's retention period (90 days by default). - Back up the metadata database like any application database. It holds encrypted connection strings and tokens; they are useless without `Beacon:EncryptionKey`, so keep the key's backup separate. - Beacon's own data access uses short-lived contexts from `IDbContextFactory`; size the pool for your job runner's worker count plus web traffic. The reference host caps Warp at `min(2 × CPU count, 20)` workers for this reason. --- # Sign-in & identity > Every way a person or program proves who it is to Beacon: first-run setup, local passwords and break-glass accounts, OIDC/Entra SSO with MFA, bearer JWTs, API keys, sessions and your own sign-in. Source: https://moberghr.github.io/beacon/integrate/authentication/ Beacon recognises five kinds of caller. You can combine them. | Caller | How it authenticates | Typical use | |---|---|---| | Person, password | Beacon's login form against Beacon's user store | Small teams, break-glass | | Person, SSO | OIDC code flow (Entra ID or any OIDC provider) | Organisations with an identity provider | | Person, your app | Your host's own authentication scheme | Beacon embedded inside your product | | Program, API key | `Authorization: Bearer sk-sem_…` | Scripts, CI, MCP clients | | Program or delegated user, JWT | `Authorization: Bearer ` | Entra-authenticated MCP clients, service-to-service | ## Beacon's user store and first-run setup `options.EnableUserManagement()` keeps users and roles in Beacon's database. While no administrator exists: 1. Every browser request is redirected to `/beacon/setup`. 2. The setup page asks for a **setup token**: `Beacon:UserManagement:SetupToken` when set (at least 32 characters), otherwise a random token Beacon prints **once to the process console (stderr)** at startup, never to the logger. 3. Whoever presents the token creates the first super admin. `POST /beacon/api/setup/superadmin` is anonymous but throttled per IP; once an administrator exists, setup is closed for good. :::caution With several replicas, each one generates its own console token. Set `Beacon:UserManagement:SetupToken` from your secret store for multi-instance deployments, and remove it once setup is done. ::: Until the first administrator exists, SSO and MCP sign-ins cannot provision other users, and every request except setup, the health check (`/beacon/api/health`) and the MCP discovery documents is redirected to `/beacon/setup`. ### Headless first run (scripts and CI) The setup token appears on the host's console as `Beacon first-run setup token (enter it on the setup page): `. ```bash BEACON=https://localhost:7187 curl -s $BEACON/beacon/api/setup/status curl -s -X POST $BEACON/beacon/api/setup/superadmin \ -H "Content-Type: application/json" \ -d '{"setupToken":"","userName":"admin","displayName":"Admin","email":"admin@example.com","password":"","confirmPassword":""}' # Sign in with an antiforgery token (login needs one too); keep the cookie jar for later calls. TOKEN=$(curl -s -c jar.txt $BEACON/beacon/api/csrf | jq -r .token) curl -s -b jar.txt -c jar.txt -X POST $BEACON/beacon/api/auth/login \ -H "Content-Type: application/json" -H "X-XSRF-TOKEN: $TOKEN" \ -d '{"username":"admin","password":"","rememberMe":false}' curl -s -b jar.txt $BEACON/beacon/api/auth/me ``` State-changing calls with the cookie need an antiforgery token: `GET /beacon/api/csrf` returns `{"token":"…"}` and sets a cookie; send the token as `X-XSRF-TOKEN` **with the same cookie jar**. The token is bound to the signed-in user, so fetch a new one after login: ```bash TOKEN=$(curl -s -b jar.txt -c jar.txt $BEACON/beacon/api/csrf | jq -r .token) # A project (you become its ProjectAdmin) … curl -s -b jar.txt -c jar.txt -X POST $BEACON/beacon/api/projects \ -H "Content-Type: application/json" -H "X-XSRF-TOKEN: $TOKEN" \ -d '{"name":"Acme Shop","description":"Shop data"}' # → {"projectId":1,…} # … and an Execute-scoped API key for it (the key is returned once). curl -s -b jar.txt -c jar.txt -X POST $BEACON/beacon/api/api-keys \ -H "Content-Type: application/json" -H "X-XSRF-TOKEN: $TOKEN" \ -d '{"name":"ci-mcp","scopes":["Execute"],"allowedProjectIds":[1]}' # → {"plainTextKey":"sk-sem_…",…} ``` The full request shapes are in the OpenAPI document (`/openapi/v1.json`), once setup is complete. ### Passwords Passwords are hashed with PBKDF2-SHA256 (600,000 iterations stored in the hash, 32-byte salt; older hashes are upgraded at the next successful sign-in, see [Password hashes](https://moberghr.github.io/beacon/security/user-management/#password-hashes)). Policy is code-configured: `MinimumPasswordLength` (default 8) and `RequirePasswordComplexity` (default `true`). A password an administrator sets must be changed at first sign-in. Users change their own password under **Settings**; there is no REST route that sets another user's password. Password sign-in is throttled to **10 attempts per 60 seconds per client IP** (429 with `Retry-After`), and an internal user is locked out for 15 minutes after 5 wrong passwords (the same generic answer, whatever password is sent; break-glass accounts are never locked, and `Beacon:RateLimits:Lockout:Enabled` `false` turns the lockout off). IPv6 clients are limited per `/64` block. Behind a proxy the per-IP limit only works if your host runs `UseForwardedHeaders` with the proxy listed. See [Production checklist](https://moberghr.github.io/beacon/integrate/production/#reverse-proxy) and [Request limits](https://moberghr.github.io/beacon/getting-started/configuration/#request-limits). ### Turning password sign-in off (break-glass) With SSO in place, turn local passwords off and keep up to three named break-glass accounts: ```json { "Beacon": { "UserManagement": { "LocalLogin": { "Enabled": false, "BreakGlassUserNames": [ "beacon-breakglass" ] } } } } ``` - `Enabled: false` unmaps `POST /beacon/api/auth/login` (unless break-glass names exist), hides the password form, and makes every password provider refuse. - `BreakGlassUserNames`: at most three **stored user names**, matched exactly and case-sensitively, enabled and not archived. Each use logs `BreakGlassSignIn` (event 7119, Warning). Treat every occurrence as an incident. - Startup fails if local login is off and nobody else could sign in (no SSO, no external login, no break-glass name). ## OIDC / SSO ```csharp builder.Services.AddBeaconOidcAuthentication(builder.Configuration); ``` ```json { "Beacon": { "Authentication": { "Oidc": { "Enabled": true, "Authority": "https://login.microsoftonline.com//v2.0", "ClientId": "", "ClientSecret": "", "AllowedTenants": [ "" ], "RequiredGroups": [ "" ], "DefaultRoleName": "Viewer", "RequireMfa": true } } } } ``` | Key | Default | What it does | |---|---|---| | `Enabled` | `false` | Master switch. When on, startup validates the section. | | `Authority`, `ClientId`, `ClientSecret` | Required | Required when enabled. Code flow with PKCE; tokens are not saved in the cookie. Keep the secret in a secret store. | | `CallbackPath` | `/signin-oidc` | Redirect URI path to register with the identity provider. | | `Scopes` | `openid profile email` | Requested scopes (replaces the list). | | `AllowedTenants` | None | Entra `tid` values that may sign in. **Required** unless `AllowAnyTenant`. | | `AllowAnyTenant` | `false` | Skip the tenant check (identity providers without `tid`). Warns at startup with an Entra authority. | | `BlockGuests` | `true` | Refuse Entra B2B guests (`acct=1`, or `idp` naming another provider). | | `RequiredGroups` / `RequiredRoles` | None | When either is set, the user needs at least one listed group object id or app role. | | `DefaultRoleName` | none | Role given to a user created at first SSO sign-in. Unset: new users get **no** role until someone grants one. Cannot be `Admin` or a functional role. | | `RequireMfa` | `false` | Refuse sign-ins whose ID token has no `amr` containing `mfa` and no `acr` listed in `AcceptedAcrValues`. | | `AcceptedAcrValues` | None | `acr` values that count as MFA. | | `EnableFrontChannelLogout` | `false` | Serve `/signout-oidc` for identity-provider-initiated logout. | | `DisplayName` | `SSO` | Not used. The login page shows a fixed "Continue with single sign-on" button, and the SSO config endpoint returns only `enabled` and `localLoginEnabled`. | Admission runs **before** any user lookup: tenant → guest block → required group or role. MFA covers the interactive SSO sign-in only. It does not apply to bearer tokens, API keys, MCP callers or break-glass passwords. Entra app roles can also grant Beacon's [functional permissions](https://moberghr.github.io/beacon/integrate/access-control/#3-functional-permissions) at sign-in (`Beacon:Authorization:IdpRoles`). ## Bearer JWTs Bearer access tokens work for the REST API and for MCP. They are configured in code: ```csharp builder.Services.AddBeaconJwtAuthentication(jwt => { jwt.EnableBearerAuthentication = true; jwt.Validation.JwksEndpoint = "https://login.microsoftonline.com//discovery/v2.0/keys"; jwt.Validation.ValidIssuers = [ "https://login.microsoftonline.com//v2.0", "https://sts.windows.net//" ]; jwt.Validation.ValidAudiences = [ "api://", "" ]; }); // … app.UseBeaconAuthentication(); // judges the bearer token, the API key or the cookie, whichever is presented ``` | Option | Default | Notes | |---|---|---| | `EnableBearerAuthentication` | `false` | Turns the middleware on. | | `Validation.JwksEndpoint` or `Validation.SigningKey` | Required | One is required. The JWKS endpoint is an `https` URL (`http` to a loopback host in Development only); a signing key is at least 32 bytes. | | `Validation.ValidIssuer(s)`, `ValidAudience(s)` | Required | At least one of each; issuer, audience and lifetime validation must stay on. | | `Validation.ClockSkew` | 5 min | At most 5 minutes. | | `AccessTokenClaim` | None | For non-Entra providers: a claim that proves the token is an access token (ID-token claims such as `sub` or `nonce` are refused). | | `ClaimsMapping` | `sub`, `preferred_username`, `email`, `roles`, `name` | Which claims carry the user id, user name, email, roles and display name. | | `ExternalLoginEndpoint` | None | Let the login form validate credentials against your own token endpoint (an `https` URL). Internal users still sign in first. | A token must prove it is an **access** token (Entra: `azp`/`appid` plus `scp` or `roles`; others: `typ: at+jwt` or `AccessTokenClaim`) and must bind to an existing, enabled Beacon user, so bearer tokens on the REST API need Beacon's user management. On `/beacon/mcp`, tokens go through the **MCP caller mapping** instead (`Beacon:Mcp:Callers`). See [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/#option-b-entra-id-callers). ## API keys Users create API keys under **API keys** (`/beacon/api-keys`). A key: - is `sk-sem_` followed by about 43 random base64 characters, shown **once**, stored as a SHA-256 hash; - has the **Read** or **Execute** scope (Read: reads only; Execute: everything the owner may do, including running SQL and MCP); - names **at least one project** the owner can read, and reaches only those; - expires: 90 days by default, at most `Beacon:ApiKeys:MaxLifetimeDays` (365); - acts as its owner, capped by the lower of its scope and the owner's current role, never with the Admin bypass and never with a functional permission; - stops working when its owner is disabled or archived. Send it as `Authorization: Bearer sk-sem_…`. Details: [API keys](https://moberghr.github.io/beacon/security/api-keys/). ## Sessions | Setting | Default | What it does | |---|---|---| | `Beacon:Authentication:Session:RevalidateMinutes` | `5` | How often a session is re-checked against the user store (user enabled, not archived, unchanged security stamp; roles reloaded). `0` checks every request. | | `Beacon:Authentication:Session:AbsoluteLifetimeHours` | `12` | A session ends this long after sign-in, however often it is renewed. "Remember me" cannot extend it. | | `Beacon:Authentication:Session:PersistSsoCookie` | `false` | A single sign-on session gets a browser-session cookie, dropped when the browser closes. `true` keeps the cookie until it expires (still within `AbsoluteLifetimeHours`). Password sign-in follows "remember me". | The `Beacon.Auth` cookie is HttpOnly, SameSite=Lax and Secure on HTTPS. Changing a user's roles, password, enabled state or archiving them renews their **security stamp**, which ends every session they have. Users can sign out everywhere themselves; a UserAdmin can do it for them (`POST /beacon/api/users/{id}/sign-out-everywhere`). Realtime connections close when the session ends, and the UI stops its connection and drops its cached data when it signs out or gets a `401`. ## Your app owns sign-in When Beacon runs inside your product and your users already sign in to it: 1. Leave `EnableUserManagement()` and `EnableLoginForm` off. 2. Register your authentication scheme as usual; keep `AddBeaconCookieAuthentication("/beacon")` only if you also want the Beacon cookie. 3. Make your principal carry `ClaimTypes.NameIdentifier` and a role claim of `Admin`, `Editor` or `Viewer` (your claims transformation can map your roles). 4. Tell the UI where to send signed-out users: ```csharp builder.Services.AddBeaconApiServices(x => x.ExternalLogin = new ExternalLoginOptions { LoginUrl = "/account/login" }); ``` Without Beacon's user store: - project membership comes from an `IProjectMembershipSource` you register, or projects stay open; - only the `Admin` role claim grants functional permissions, unless `Beacon:Authorization:Roles:HonourHostRoleClaims` is `true`, in which case a role claim named like a permission's role grants it. ## Choosing a combination | Deployment | Recommended | |---|---| | Internal tool, small team | User store + login form; one break-glass account. | | Company with Entra ID | User store + OIDC (`AllowedTenants`, `RequiredGroups`, `RequireMfa`), local login off with one break-glass name, IdP app roles for functional permissions. | | Embedded in your SaaS | Your sign-in + `ExternalLogin` + a membership source; Beacon cookie optional. | | AI clients | API keys per person (Execute scope, one project each), or Entra callers for organisation-wide MCP access. | --- # Access control > Decide who sees what before go-live: global roles, projects as the access boundary, functional permissions, separation of duties, IdP role mapping, change approvals and deployment locks. Source: https://moberghr.github.io/beacon/integrate/access-control/ Beacon's access model has four layers. Set each one deliberately before people start using it. ```mermaid flowchart TB R["Global role
Admin · Editor · Viewer
the ceiling"] --> P["Project membership
ProjectAdmin · Editor · Viewer
what you can touch"] P --> A["Project action
Read · Run · Query · Write · Manage · ManageMembers · Curate"] F["Functional permissions
Approver · Auditor · SecurityAdmin
UserAdmin · DataSourceAdmin · MigrationOperator
duties, no data"] -.-> X["Admin screens and decisions"] L["Deployment locks
config only, bind Admins too"] -.-> A ``` ## 1. Global roles Every user has at most one global role, which caps everything else: | Role | Can | |---|---| | **Viewer** | Read, and run existing checks, in projects they can reach. | | **Editor** | Also create and change queries, subscriptions and other project content. | | **Admin** | Everything, in every project, plus every functional permission. | The first super admin is created at [first-run setup](https://moberghr.github.io/beacon/integrate/authentication/#beacons-user-store-and-first-run-setup). Give `Admin` to as few people as possible. The functional permissions below exist so that most administrative duties do not need it. ## 2. Projects are the access boundary A **project** groups data sources and the queries, subscriptions, contracts and grounding built on them. Access is decided per project by one component, `IProjectAccessEvaluator`: - **Restricted** projects are visible only to their **members** and to Admins. Everyone else gets **404**, so the existence of the project is not disclosed. - **Open** projects are visible to everyone with a global role, at that role. - **New projects start restricted**, with their creator as **ProjectAdmin**. - A **data source follows the projects that link it**. Something that reads several data sources is reachable only through one project that links all of them. Data sources created after project access was introduced are Admin-only until linked to a project. | Project role | Actions | |---|---| | Viewer | Read, Run | | Editor | + Query (ad-hoc SQL, MCP queries), Write; + Manage in open projects | | ProjectAdmin | + Manage, ManageMembers, Curate (review learned patterns, documentation and relationships) | A global **Viewer** is a Viewer in every project, whatever their membership; Editors and Admins act with their membership role (and in an open project, with the higher of that and their global role). `ManageMembers` and `Curate` come only from a ProjectAdmin membership and are never granted to API keys, MCP callers or scheduled work. **Make every project restricted** with `Beacon:Authorization:RequireProjectMembership: true`. Startup then requires a membership source (Beacon's user store, or your `IProjectMembershipSource`). Plan the first projects before inviting users: 1. Create one project per team or data domain. 2. Link each data source to the projects that should see it. 3. Add members with the lowest role that works. ## 3. Functional permissions Administrative duties are split out of the Admin role. Each is held through a **functional role** of the same name (a system role with no data access), assigned by an Admin, or mapped from an identity-provider app role at SSO sign-in. | Permission | Grants | |---|---| | **Approver** | Approve or reject query changes and change requests (migration jobs, Custom SQL rules, recipient destinations and, when covered, data-source connections). A change of the LLM connection is an Admin's to decide. | | **Auditor** | Read and verify the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/); read the MCP audit log, MCP settings, admin settings, every API key and what a user owns, created or changed last. | | **SecurityAdmin** | Change MCP and admin settings (not the LLM connection or base URL, which are Admin only), manage notification recipients, publish saved queries as MCP tools, set repository tokens. | | **UserAdmin** | Manage users and their non-privileged roles, end sessions, see what a user owns, created or changed last, revoke API keys, change owners of scheduled work, manage project members, restrict projects. | | **DataSourceAdmin** | Create, test and delete data sources; link them to projects they can read. | | **MigrationOperator** | Create and run data-migration jobs (only when migrations are enabled). | A functional permission grants **no project data**. A UserAdmin can never grant Admin or a functional role, nor touch Admins or other privileged users. API keys never hold a functional permission. ```json { "Beacon": { "Authorization": { "RequireProjectMembership": true, "Roles": { "Approver": "Beacon Approver" }, "IdpRoles": { "Enabled": true, "Map": { "Beacon.Auditor": "Auditor", "Beacon.SecurityAdmin": "SecurityAdmin" }, "ClaimLifetimeMinutes": 60 }, "SeparationOfDuties": { "Enabled": true } } } } ``` | Key | Default | What it does | |---|---|---| | `Roles:{Permission}` | the permission's name | Rename the functional role that grants a permission (not Admin/Editor/Viewer; unique). | | `Roles:HonourHostRoleClaims` | `false` | Without Beacon's user store: let a role claim named like a functional role grant the permission. | | `IdpRoles:Enabled` / `Map` | off | Map identity-provider app roles to permissions at SSO sign-in. Mapped permissions live in the session only and are not stored. | | `IdpRoles:ClaimLifetimeMinutes` | `60` | How long a mapped permission counts after sign-in. | | `SeparationOfDuties:Enabled` | `false` | Refuse giving one person both halves of a conflicting pair: Approver + SecurityAdmin, Auditor + SecurityAdmin, Auditor + UserAdmin, UserAdmin + Approver, UserAdmin + MigrationOperator. | ## 4. Change approvals (four-eyes) Opt in with `options.EnableApprovalWorkflow(...)`. Then a change to a query's executable content goes live only after a **second person** with the Approver permission approves exactly the content they reviewed, and scheduled runs execute only reviewed SQL. By default it also covers migration jobs, Custom SQL rules and new or changed notification destinations; data-source connections (`CoversDataSourceConnections`), the LLM connection (`CoversLlmConnection`) and grants of the Admin role (`CoversAdminRoleGrants`) are covered when you switch them on. ```csharp options.EnableApprovalWorkflow(x => { x.RequireApprovedVersionToRun = true; // strict: run only versions with an approval for their exact content // x.AllowSelfApproval = true; // only for a deployment with a single possible approver // x.CoversMigrationJobs = false; // per-kind switches, all true by default }); ``` Make sure at least two people hold Approver (Admins count). Full rules: [Approvals](https://moberghr.github.io/beacon/security/approvals/). ## 5. Deployment locks Some guarantees should not depend on an individual Admin's actions. These keys can only be changed in host configuration. The UI shows the setting disabled, and the API answers 409: | Key | Default | Pins | |---|---|---| | `Beacon:Mcp:ForceReadOnly` | `true` | Read-only SQL everywhere the gate runs. | | `Beacon:Mcp:ForcePiiDetection` | `true` | PII detection and masking on. | | `Beacon:Mcp:ForceNoContentRetention` | `false` | No question, SQL or error text stored with MCP audit rows. | | `Beacon:Mcp:ForbidRowDataToLlm` | `false` | No row values (samples, value grounding, eval judging) sent to the LLM provider. | | `Beacon:Mcp:Audit:RequireContent` | `false` | Content retention on: MCP audit rows keep the question and the SQL of every statement a call ran. Cannot be combined with `ForceNoContentRetention`. | | `Beacon:Mcp:Ceilings:MaxRowLimit` | none | Upper bound for every project's MCP row limit. | | `Beacon:Migrations:Enabled` | `false` | The only write path. Off: no migration endpoints, no jobs, for anyone. | See [Security model](https://moberghr.github.io/beacon/security/) for how these fit together, and [Authorization](https://moberghr.github.io/beacon/security/authorization/) for the full reference. ## A starting configuration For a typical company deployment: - `RequireProjectMembership: true` - SSO with `AllowedTenants`, `RequiredGroups`, `RequireMfa`; local login off with one break-glass account - Two or more Approvers, one Auditor, one SecurityAdmin; `SeparationOfDuties` on - Approval workflow on, with `RequireApprovedVersionToRun` - Locks left at their defaults; `ForbidRowDataToLlm` on if row values must never reach your LLM provider - Migrations off unless you need them - `Beacon:Audit:IntegrityKey` set from a secret store, and the security audit verification and retention jobs scheduled --- # Background jobs > Beacon runs no background work of its own; your job runner does. The IBeaconScheduler contract, the jobs Beacon expects, the system-work rule, a complete Warp implementation, and a Hangfire sketch. Source: https://moberghr.github.io/beacon/integrate/background-jobs/ Scheduled checks, data-quality evaluations, documentation generation, AI actor cycles and maintenance all run as background jobs. Beacon defines **what** must run; your host decides **how**, through one interface: `IBeaconScheduler`. ```mermaid sequenceDiagram participant Beacon as Beacon (handlers) participant Sched as Your IBeaconScheduler participant Runner as Your job runner participant Jobs as IJobService (Beacon) Beacon->>Sched: AddOrUpdate(subscriptionId, name, cron) Sched->>Runner: register recurring job "subscription-{id}" Note over Runner: cron fires Runner->>Runner: new DI scope, mark as system work Runner->>Jobs: ExecuteQuery(subscriptionId) Jobs->>Jobs: run as the subscription's owner, re-check access, gate SQL, notify ``` ## The contract ```csharp namespace Beacon.Core.Worker; public interface IBeaconScheduler { Task AddOrUpdate(int subscriptionId, string subscriptionName, string cron); Task Remove(int subscriptionId, string subscriptionName); // The members below have default bodies that throw NotImplementedException. // Implement them to enable the feature that calls them. // EnqueueMetadataRefresh and EnqueueBackgroundTask are the exceptions: their defaults return null instead of throwing. Task AddOrUpdateDataQualityJob(int contractId, string contractName, string cron); // data contracts Task RemoveDataQualityJob(int contractId, string contractName); Task EnqueueProjectDocumentation(int projectId, int userId, string? notifyUserId); // AI documentation Task EnqueueAiActorThinkCycle(int actorId, int subscriptionId); // AI actors Task EnqueueMcpEval(int runId); // MCP evaluation runs Task EnqueueMetadataRefresh(int dataSourceId); // metadata re-scan after a connection target change Task EnqueueBackgroundTask(int taskId, string? notifyUserId); // a task a user started and follows (metadata refresh, migration run, repository scan) } ``` `EnqueueBackgroundTask` is optional too. Beacon records long work a user starts from the UI (a forced metadata refresh, an AI actor's think-now, a manual migration run, a scan of a project's repositories, a project's documentation generation) as a background task and hands its id to this method; your job calls `IBackgroundTaskService.ExecuteAsync(taskId, ct)` in a scope marked as system work; Beacon then runs it as the stored user who started it (by `Users.Id`) and re-checks their access first (a task they may no longer run is recorded as skipped, and the job succeeds). It runs with the user's own access, not the credential's: a start made with an API key is not capped by the key's scope or projects in the job, and revoking or narrowing the key after the start does not stop it (disabling the user, or removing its access, does). Return the job id (the UI follows `JobStatusChanged` for it, sent to `notifyUserId`: the id the SignalR hub addresses the caller by, a signed-in session's external id, or `null` for an API key or an MCP caller), or `null` (the default): Beacon then runs the task in the request that started it, as before. `EnqueueMetadataRefresh` is optional. Beacon calls it after a data source's connection is pointed at another target, to re-scan its metadata in the background. Return the job id, or `null` (the default) when your host runs no such job; Beacon then logs that the old schema stays until a manual refresh. Register it with `options.AddBeaconScheduler()`. Without one, `AddBeaconServices` throws `Implementation of IBeaconScheduler is required.` - `cron` is a standard **5-field** expression (validated by Beacon before it reaches you). - Key recurring jobs by the **entity id** (`subscription-{id}`, `dq-{id}`), never by the name. Names change, ids do not. - The `Enqueue…` methods return your runner's job id as a string. The UI correlates live `JobStatusChanged` events by that id; `notifyUserId` is the user who should receive them. ## What each job calls Each job opens a **new DI scope**, marks it as system work, and calls one Beacon method: | Job | Calls | Scheduled by | |---|---|---| | Subscription run | `IJobService.ExecuteQuery(subscriptionId, ct)` | `AddOrUpdate` / `Remove` | | Data contract evaluation | `IJobService.EvaluateDataContract(contractId, ct)` | `AddOrUpdateDataQualityJob` / `RemoveDataQualityJob` | | Project documentation (5.0 route and job) | `IProjectDocumentationService.GenerateDocumentationAsync(projectId, userId, ct)` (Beacon.AI) | `EnqueueProjectDocumentation` | | AI actor think cycle | `AiActorService.ExecuteThinkCycleBackgroundAsync(actorId, subscriptionId)` (Beacon.AI) | `EnqueueAiActorThinkCycle` | | MCP evaluation run | `IJobService.RunMcpEval(runId, ct)` | `EnqueueMcpEval` | | Background task (metadata refresh, AI actor think-now, migration run, repository scan, documentation generation) | `IBackgroundTaskService.ExecuteAsync(taskId, ct)` | `EnqueueBackgroundTask` | ### Recurring maintenance jobs Register these yourself at startup (the reference host's schedule): | Id | Cron | Calls | Purpose | |---|---|---|---| | `mcp-learning-aggregate` | `0 */6 * * *` | `IJobService.AggregateLearnedPatterns` | Mine MCP usage signals into learned patterns. | | `mcp-learning-cleanup` | `0 3 * * *` | `IJobService.CleanupOldSignals` | Delete signals past their retention. | | `mcp-audit-retention` | `30 3 * * *` | `IMcpAuditRetentionService.PurgeExpiredAsync` | Purge MCP audit rows older than `Beacon:Mcp:Audit:RetentionDays` (no-op when unset or `0`). | | `data-retention` | `50 3 * * *` | `IDataRetentionService.RunAsync` | Delete run history and logs older than `Beacon:Retention:Days` (no-op while it and every target are `0`, the default) and clear content older than `Beacon:Retention:ContentDays` (content first). A run works at most 50 minutes, each step within its share, and the next run continues what it left. `[Timeout]` one hour, `[Mutex]` one run at a time. See [Data retention](https://moberghr.github.io/beacon/getting-started/configuration/#data-retention-beaconretention). | | `security-audit-retention` | `45 3 * * *` | `ISecurityAuditRetentionService.PurgeExpiredAsync` | Purge security audit events older than `Beacon:Audit:RetentionDays` (no-op when unset: events are kept forever). | | `security-audit-verify` | `15 * * * *` | `ISecurityAuditVerificationService.VerifyNewEventsAsync` | Check events, then MCP audit, execution log and settings history rows, written since the last run against their integrity hashes; failures are logged at Error. | | `mcp-embedding-reindex` | `0 */12 * * *` | `IJobService.ReindexEmbeddings` | Re-embed metadata and examples for semantic retrieval. Content that did not change since the last run (same text, model and version) is skipped, so a run costs what changed. | | `mcp-docchunk-reindex` | `0 */12 * * *` | `IJobService.ReindexDocChunks` | Re-chunk and re-embed project documentation. | One-off operator jobs: - While `Beacon:PreviousEncryptionKeys` lists a key, run `IStoredValueReEncryptionService.ReEncryptAsync` once per start (the reference host checks `Beacon.Core.ServiceConfiguration.HasPreviousEncryptionKeys(configuration)`). Remove the old key when it reports nothing left. - `IRecipientSecretEncryptionService.EncryptStoredSecretsAsync` encrypts recipient secrets stored before encryption existed. Run it once, by hand, after **every** instance runs the new version. ## The system-work rule Beacon decides project access from the caller. A background job has no HTTP request, so it must say that it is Beacon's own work: ```csharp scope.ServiceProvider.GetRequiredService().MarkAsSystemWork(); // Beacon.Core.Authorization ``` Do this in the job's scope **before** Beacon code runs. An unmarked scope without a request gets **no project access** and logs a warning, and the job does nothing useful. Work done for a user (a subscription's scheduled run) is still executed **as that user**: Beacon sets the run-as owner itself and re-checks their access to the exact data sources right before any SQL or LLM call. A run the owner may no longer do is recorded as skipped and never run as the system. Do not resolve the current user in your job wrapper before Beacon's method runs. ## Reference implementation: Moberg Warp [Moberg Warp](https://moberghr.github.io/warp/) is a PostgreSQL-backed job runner with recurring jobs and a dashboard. The reference host (`src/Beacon.SampleProject`) uses Warp 3.11; the code below matches it. Check the Warp docs if you use a newer major version. ```bash dotnet add package Moberg.Warp.Core --version 3.11.0 dotnet add package Moberg.Warp.Worker --version 3.11.0 dotnet add package Moberg.Warp.Provider.PostgreSql --version 3.11.0 dotnet add package Moberg.Warp.UI --version 3.11.0 # optional dashboard dotnet add package EFCore.NamingConventions # UseSnakeCaseNamingConvention dotnet add package Microsoft.EntityFrameworkCore.Design # for dotnet ef (same EF Core 10 version) dotnet tool install --global dotnet-ef ``` **1. A dedicated context for Warp's tables** (the `warp` schema), kept apart from Beacon's: ```csharp using Microsoft.EntityFrameworkCore; public sealed class WarpDbContext(DbContextOptions options) : DbContext(options); ``` Create its initial migration once in your project: `dotnet ef migrations add WarpInitial --context WarpDbContext`. Warp supplies the model. **2. Registration and startup:** ```csharp using Microsoft.EntityFrameworkCore; using Warp.Core; using Warp.Provider.PostgreSql; using Warp.Worker; public static class BeaconWarpSetup { public static IServiceCollection AddBeaconWarp(this IServiceCollection services, IConfiguration configuration) { // Warp supports PostgreSQL; with Beacon on SQL Server, point WarpContext at a PostgreSQL database. services.AddDbContext(x => x .UseNpgsql(configuration.GetConnectionString("WarpContext") ?? configuration.GetConnectionString("BeaconContext")) .UseSnakeCaseNamingConvention()); services.AddWarpServer(x => { x.UsePostgreSql(); x.PollingInterval = TimeSpan.FromSeconds(1); x.WorkerCount = Math.Min(Environment.ProcessorCount * 2, 20); // protects the connection pool // No x.AddRetry(): failed Beacon jobs are investigated, not retried blindly. }); return services; } public static async Task StartBeaconWarpAsync(this WebApplication app) { using var scope = app.Services.CreateScope(); scope.ServiceProvider.GetRequiredService().Database.Migrate(); var recurring = scope.ServiceProvider.GetRequiredService(); await recurring.AddOrUpdateRecurringJob(new AggregateLearnedPatternsJob(), "mcp-learning-aggregate", "0 */6 * * *"); await recurring.AddOrUpdateRecurringJob(new CleanupOldSignalsJob(), "mcp-learning-cleanup", "0 3 * * *"); await recurring.AddOrUpdateRecurringJob(new PurgeExpiredMcpAuditLogsJob(), "mcp-audit-retention", "30 3 * * *"); await recurring.AddOrUpdateRecurringJob(new PurgeExpiredSecurityAuditEventsJob(), "security-audit-retention", "45 3 * * *"); await recurring.AddOrUpdateRecurringJob(new DataRetentionJob(), "data-retention", "50 3 * * *"); await recurring.AddOrUpdateRecurringJob(new VerifySecurityAuditEventsJob(), "security-audit-verify", "15 * * * *"); await recurring.AddOrUpdateRecurringJob(new ReindexEmbeddingsJob(), "mcp-embedding-reindex", "0 */12 * * *"); await recurring.AddOrUpdateRecurringJob(new ReindexDocChunksJob(), "mcp-docchunk-reindex", "0 */12 * * *"); } } ``` **3. Mark every job as system work** with a pipeline behaviour (Warp's source generator registers it): ```csharp using Beacon.Core.Authorization; using Warp.Core.Handlers; public sealed class SystemWorkBehavior(ISystemWorkScope systemWork) : IPipelineBehavior where TRequest : IRequest { public Task HandleAsync(TRequest request, RequestHandlerDelegate next, CancellationToken cancellationToken) { systemWork.MarkAsSystemWork(); return next(request, cancellationToken); } } ``` **4. One job + handler per Beacon entry point:** ```csharp using Beacon.Core.SecurityAudit; using Beacon.Core.Services.Retention; using Beacon.Core.Worker; using Warp.Core.Handlers; public sealed class ExecuteSubscriptionQueryJob : IJob { public int SubscriptionId { get; init; } } public sealed class ExecuteSubscriptionQueryJobHandler(IJobService jobs) : IJobHandler { public Task HandleAsync(ExecuteSubscriptionQueryJob message, CancellationToken cancellationToken) => jobs.ExecuteQuery(message.SubscriptionId, cancellationToken); } public sealed class EvaluateDataContractJob : IJob { public int ContractId { get; init; } } public sealed class EvaluateDataContractJobHandler(IJobService jobs) : IJobHandler { public Task HandleAsync(EvaluateDataContractJob message, CancellationToken cancellationToken) => jobs.EvaluateDataContract(message.ContractId, cancellationToken); } public sealed class RunMcpEvalJob : IJob { public int RunId { get; init; } } public sealed class RunMcpEvalJobHandler(IJobService jobs) : IJobHandler { public Task HandleAsync(RunMcpEvalJob message, CancellationToken cancellationToken) => jobs.RunMcpEval(message.RunId, cancellationToken); } public sealed class AggregateLearnedPatternsJob : IJob; public sealed class AggregateLearnedPatternsJobHandler(IJobService jobs) : IJobHandler { public Task HandleAsync(AggregateLearnedPatternsJob message, CancellationToken cancellationToken) => jobs.AggregateLearnedPatterns(cancellationToken); } public sealed class CleanupOldSignalsJob : IJob; public sealed class CleanupOldSignalsJobHandler(IJobService jobs) : IJobHandler { public Task HandleAsync(CleanupOldSignalsJob message, CancellationToken cancellationToken) => jobs.CleanupOldSignals(cancellationToken); } public sealed class ReindexEmbeddingsJob : IJob; public sealed class ReindexEmbeddingsJobHandler(IJobService jobs) : IJobHandler { public Task HandleAsync(ReindexEmbeddingsJob message, CancellationToken cancellationToken) => jobs.ReindexEmbeddings(cancellationToken); } public sealed class ReindexDocChunksJob : IJob; public sealed class ReindexDocChunksJobHandler(IJobService jobs) : IJobHandler { public Task HandleAsync(ReindexDocChunksJob message, CancellationToken cancellationToken) => jobs.ReindexDocChunks(cancellationToken); } public sealed class PurgeExpiredMcpAuditLogsJob : IJob; public sealed class PurgeExpiredMcpAuditLogsJobHandler(IMcpAuditRetentionService retention) : IJobHandler { public Task HandleAsync(PurgeExpiredMcpAuditLogsJob message, CancellationToken cancellationToken) => retention.PurgeExpiredAsync(cancellationToken); } public sealed class PurgeExpiredSecurityAuditEventsJob : IJob; public sealed class PurgeExpiredSecurityAuditEventsJobHandler(ISecurityAuditRetentionService retention) : IJobHandler { public Task HandleAsync(PurgeExpiredSecurityAuditEventsJob message, CancellationToken cancellationToken) => retention.PurgeExpiredAsync(cancellationToken); } // [Timeout] and [Mutex] go on the job type (Warp.Core.Timeout, Warp.Core.Concurrency), never on the handler. [Timeout(3600)] [Mutex("data-retention")] public sealed class DataRetentionJob : IJob; public sealed class DataRetentionJobHandler(IDataRetentionService retention) : IJobHandler { public Task HandleAsync(DataRetentionJob message, CancellationToken cancellationToken) => retention.RunAsync(cancellationToken); } public sealed class VerifySecurityAuditEventsJob : IJob; public sealed class VerifySecurityAuditEventsJobHandler(ISecurityAuditVerificationService verification) : IJobHandler { public Task HandleAsync(VerifySecurityAuditEventsJob message, CancellationToken cancellationToken) => verification.VerifyNewEventsAsync(cancellationToken); } ``` With AI enabled (`UseAI = true`), add the two AI jobs too: ```csharp using Beacon.AI.Services.Ai.AiActor; // AiActorService using Beacon.AI.Services.Documentation; // IProjectDocumentationService using Warp.Core.Handlers; public sealed class GenerateProjectDocumentationJob : IJob { public int ProjectId { get; init; } public int UserId { get; init; } } public sealed class GenerateProjectDocumentationJobHandler(IProjectDocumentationService documentation) : IJobHandler { public Task HandleAsync(GenerateProjectDocumentationJob message, CancellationToken cancellationToken) => documentation.GenerateDocumentationAsync(message.ProjectId, message.UserId, cancellationToken); } public sealed class AiActorThinkCycleJob : IJob { public int ActorId { get; init; } public int SubscriptionId { get; init; } } public sealed class AiActorThinkCycleJobHandler(AiActorService actors) : IJobHandler { public Task HandleAsync(AiActorThinkCycleJob message, CancellationToken cancellationToken) => actors.ExecuteThinkCycleBackgroundAsync(message.ActorId, message.SubscriptionId); } ``` The reference host's versions are in `src/Beacon.SampleProject/Warp/Jobs/`. **5. The scheduler:** ```csharp using Beacon.Core.Worker; using Microsoft.EntityFrameworkCore; using Warp.Core; using Warp.Core.Data.Entities; public sealed class BeaconScheduler( IPublisher publisher, IRecurringJobPublisher recurring, WarpDbContext warp) : IBeaconScheduler { public Task AddOrUpdate(int subscriptionId, string subscriptionName, string cron) => recurring.AddOrUpdateRecurringJob(new ExecuteSubscriptionQueryJob { SubscriptionId = subscriptionId }, $"subscription-{subscriptionId}", cron); public Task Remove(int subscriptionId, string subscriptionName) => RemoveRecurring($"subscription-{subscriptionId}"); public Task AddOrUpdateDataQualityJob(int contractId, string contractName, string cron) => recurring.AddOrUpdateRecurringJob(new EvaluateDataContractJob { ContractId = contractId }, $"dq-{contractId}", cron); public Task RemoveDataQualityJob(int contractId, string contractName) => RemoveRecurring($"dq-{contractId}"); public async Task EnqueueMcpEval(int runId) { var id = await publisher.Enqueue(new RunMcpEvalJob { RunId = runId }); await publisher.SaveChangesAsync(); return id.ToString(); } public async Task EnqueueProjectDocumentation(int projectId, int userId, string? notifyUserId) { var id = await publisher.Enqueue(new GenerateProjectDocumentationJob { ProjectId = projectId, UserId = userId }); await publisher.SaveChangesAsync(); return id.ToString(); } public async Task EnqueueAiActorThinkCycle(int actorId, int subscriptionId) { var id = await publisher.Enqueue(new AiActorThinkCycleJob { ActorId = actorId, SubscriptionId = subscriptionId }); await publisher.SaveChangesAsync(); return id.ToString(); } private async Task RemoveRecurring(string name) { var job = await warp.Set() .Where(x => x.Name == name) .FirstOrDefaultAsync(); if (job == null) { return; } warp.Set().Remove(job); await warp.SaveChangesAsync(); } } ``` **6. The dashboard (optional).** Keep it Admin-only: ```csharp var warpUi = new WarpUIOptions { RoutePrefix = "/warp" }; warpUi.Authorization = new AdminOnlyWarpFilter(); // IWarpAuthorizationFilter: authenticated + IsInRole("Admin") WarpDashboardGuard.Require(warpUi.Authorization, warpUi.RoutePrefix); app.UseWarpUI(warpUi); ``` Warp's default, with no filter, admits every request that reaches the route, and Beacon's login gate does not cover `/warp`. `WarpDashboardGuard.Require` (`Beacon.Api.Authentication`) throws when the filter it is given is null. Call it on the finished options' `Authorization` before mapping, as the sample's `UseGuardedWarpUI` does; a host that calls `UseWarpUI` directly is not checked. The filter reads the Beacon session, which applies to `/beacon` and the prefixes in `Beacon:Authentication:AdditionalPaths` only: list `/warp` there (`"Beacon": { "Authentication": { "AdditionalPaths": [ "/warp" ] } }`), or the dashboard refuses every request. ### Live job status in the UI The reference host also pushes `JobStatusChanged` (Processing / Succeeded / Failed) to the user who started a documentation job or a background task, through the SignalR hub. It tags the job with `BeaconUserId` metadata (the `notifyUserId` Beacon passes: the enqueuing user's hub user id, their `Users.ExternalId`) and a second pipeline behaviour sends the event to `Clients.User` with it; see `src/Beacon.SampleProject/Warp/JobStatusChangedBehavior.cs`. Without it everything works; the UI just does not show live progress for those jobs. ## Other job runners Any runner works if it can (1) run a job in its own DI scope, (2) register, update and remove recurring jobs by name, and (3) enqueue a one-off job and return an id. A Hangfire sketch: wrap Beacon's calls in a class that marks system work first: ```csharp using Beacon.Core.Authorization; using Beacon.Core.Worker; using Hangfire; public sealed class BeaconHangfireJobs(ISystemWorkScope systemWork, IJobService jobs) { public Task ExecuteSubscription(int subscriptionId) { systemWork.MarkAsSystemWork(); return jobs.ExecuteQuery(subscriptionId, CancellationToken.None); } public Task EvaluateContract(int contractId) { systemWork.MarkAsSystemWork(); return jobs.EvaluateDataContract(contractId, CancellationToken.None); } } public sealed class HangfireBeaconScheduler(IRecurringJobManager recurring) : IBeaconScheduler { public Task AddOrUpdate(int subscriptionId, string subscriptionName, string cron) { recurring.AddOrUpdate($"subscription-{subscriptionId}", x => x.ExecuteSubscription(subscriptionId), cron); return Task.CompletedTask; } public Task Remove(int subscriptionId, string subscriptionName) { recurring.RemoveIfExists($"subscription-{subscriptionId}"); return Task.CompletedTask; } } ``` Hangfire activates `BeaconHangfireJobs` in a scope per job when you use its ASP.NET Core integration. Turn its automatic retries off for these jobs (`[AutomaticRetry(Attempts = 0)]`), or a failed check reruns and notifies again. ## Operating notes - **No automatic retries.** A failed run is recorded and visible in the UI; investigate it rather than retrying blindly. If one of your own jobs needs retries, configure them on that job only. - **Worker count vs connection pool.** Each subscription run opens a Beacon context plus a connection to the monitored database. Cap workers so you do not exhaust the pool (the reference host uses `min(2 × CPU, 20)`). - **Several instances.** Your runner must make sure a recurring job fires once across the cluster. Warp and Hangfire do this through their database. - **A worker host without `Beacon.Api`.** A service that runs Beacon's jobs but serves no Beacon endpoints (a .NET Worker host referencing `Moberg.Beacon.Core` and a database provider only) keeps Core's request principal (`IBeaconRequestPrincipal`), which serves no request; every scope there runs off-request, so mark each job's scope as system work or set the owner it runs as. `Beacon.Api` registers `IBeaconUserContext`; such a host registers its own. --- # Data-source connectors > Register the connectors for the engines Beacon monitors, what each one supports, how read-only is enforced per engine, and how to create the read-only logins Beacon should use. Source: https://moberghr.github.io/beacon/integrate/connectors/ A **connector** teaches Beacon one kind of data source. Register the ones you need on the `AddBeaconServices(...)` builder, before `UsePostgreSql()` / `UseSqlServer()`: ```csharp using Beacon.Connector.PostgreSql; using Beacon.Connector.SqlServer; builder.Services.AddBeaconServices(builder.Configuration, options => { /* … */ }) .AddPostgreSqlConnector() .AddSqlServerConnector() .UsePostgreSql(); ``` | Engine | Package | Registration | |---|---|---| | PostgreSQL | `Moberg.Beacon.Connector.PostgreSql` | `.AddPostgreSqlConnector()` | | SQL Server | `Moberg.Beacon.Connector.SqlServer` | `.AddSqlServerConnector()` | | MySQL | `Moberg.Beacon.Connector.MySql` | `.AddMySqlConnector()` | | Azure Synapse | `Moberg.Beacon.Connector.AzureSynapse` | `.AddAzureSynapseConnector()` | | Snowflake | `Moberg.Beacon.Connector.Snowflake` | `.AddSnowflakeConnector()` | | Google BigQuery | `Moberg.Beacon.Connector.BigQuery` | `.AddBigQueryConnector()` | | Databricks | `Moberg.Beacon.Connector.Databricks` | `.AddDatabricksConnector()` | | AWS CloudWatch Logs | `Moberg.Beacon.Connector.CloudWatch` | `.AddCloudWatchConnector()` | | REST API (OpenAPI) | `Moberg.Beacon.Connector.Api` | `.AddApiConnector()` | Only registered engines appear in **New data source**. Data sources themselves (connection details, metadata options, project links) are created in the UI by a DataSourceAdmin or an Admin, and their connection strings are encrypted with `Beacon:EncryptionKey` before they are stored. See [Data sources](https://moberghr.github.io/beacon/features/data-sources/). ## What each engine supports | Engine | Metadata scan | Database-level read-only | Row cap stops the read early | Migration write target | |---|---|---|---|---| | PostgreSQL | Yes | **Yes**: `READ ONLY` transaction, one statement per batch | Yes | Yes | | MySQL | Yes | **Yes**: `START TRANSACTION READ ONLY`, batches off | Yes | Yes | | SQL Server | Yes | No: runs in a transaction that is always rolled back | Yes | Yes | | Azure Synapse | Yes | No | Yes | No | | Snowflake | Yes | No | Yes | No | | BigQuery | No | No: the connector's own parser check plus a dry run whose statement type must be `SELECT` | Yes: stops reading the job's result | No | | Databricks | No | No: the connector's own parser check and configured catalog | Yes: the statement is sent with `row_limit` | No | | CloudWatch | Log fields | n/a (Logs Insights queries, no SQL) | n/a | No | | REST API | OpenAPI endpoints | n/a (HTTP requests, no SQL) | n/a | No | Every SQL engine goes through the same [execution gate](https://moberghr.github.io/beacon/security/#6-the-sql-execution-gate) before anything runs: length and character checks, a guardrail, a dialect-aware AST validator, the cross-database rule, the refusal of the engine's catalog, statistics and query history, a row cap written into the SQL and a statement timeout. Where the database cannot refuse a write itself, the data source shows a badge and Beacon logs a warning at startup and on save. :::caution[Always use a read-only login] The gate is the main defence; database-level read-only is defence in depth. Give every data source a login that **can only read**, on every engine, including PostgreSQL and MySQL. A data-migration write target is the one exception, and it is a separate, Admin-marked data source. ::: ## Creating read-only logins **PostgreSQL** ```sql CREATE ROLE beacon_reader LOGIN PASSWORD '…'; GRANT CONNECT ON DATABASE shop TO beacon_reader; GRANT USAGE ON SCHEMA public TO beacon_reader; GRANT SELECT ON ALL TABLES IN SCHEMA public TO beacon_reader; ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO beacon_reader; ALTER ROLE beacon_reader SET default_transaction_read_only = on; ``` **SQL Server** ```sql CREATE LOGIN beacon_reader WITH PASSWORD = '…'; USE shop; CREATE USER beacon_reader FOR LOGIN beacon_reader; ALTER ROLE db_datareader ADD MEMBER beacon_reader; DENY INSERT, UPDATE, DELETE, ALTER, EXECUTE TO beacon_reader; ``` **MySQL** ```sql CREATE USER 'beacon_reader'@'%' IDENTIFIED BY '…'; GRANT SELECT, SHOW VIEW ON shop.* TO 'beacon_reader'@'%'; ``` **Snowflake, Synapse, BigQuery, Databricks:** a role or service principal with read access to the objects Beacon should see, and nothing else (Snowflake: a role with `USAGE` + `SELECT`; BigQuery: `BigQuery Data Viewer` + `BigQuery Job User` on the project; Databricks: `SELECT` + `USE CATALOG`/`USE SCHEMA` through Unity Catalog). ## Connection details per engine | Engine | The **New data source** dialog asks for | |---|---| | PostgreSQL, SQL Server, MySQL, Synapse, Snowflake | Database engine and connection string, plus metadata options: max tables, max columns per table, include and exclude schemas | | Databricks | Host, HTTP path, personal access token; catalog and schema (optional) | | BigQuery | Project id, service-account JSON; dataset id and location (optional); `MaximumBytesBilled` and `QueryTimeoutSeconds` in the connection settings (optional) | | CloudWatch | AWS region, log groups; access key id, secret access key or AWS profile (optional) | | REST API | Base URL, OpenAPI spec URL, authentication (API key in a header or parameter, bearer token, or basic), include and exclude path patterns | The last step of the dialog runs **Test connection** before **Create data source**. ### Cost and time limits - **BigQuery:** every query job is sent with `MaximumBytesBilled`: the data source's own value, or 10 GiB when it sets none, lowered by the deployment's `Beacon:DataSources:BigQuery:MaxBytesBilled` when that is set. A job that would bill more fails without a charge. Set the per-source value (bytes) as `MaximumBytesBilled` in the connection JSON. A job still running when the timeout passes or the caller cancels is cancelled. - **Databricks:** the statement is submitted without waiting, so a caller's cancellation or the timeout cancels it on the warehouse. Results are limited to 16 MiB (a larger one fails) and, on Beacon's capped reads, to the row cap. - **BigQuery, Databricks:** a result read without a row cap fails above 10,000 rows instead of being cut short. - **Snowflake:** each statement also runs under the session's `STATEMENT_TIMEOUT_IN_SECONDS` (`Beacon:Query:StatementTimeoutSeconds`), reset after the read; a session whose reset fails is closed without returning to the driver's pool. - **BigQuery, Databricks, CloudWatch:** a stored `QueryTimeoutSeconds` above 900 seconds is treated as 900. - **REST API:** `TimeoutSeconds` is clamped to 1–120 seconds; one response may hold 10 MB; the OpenAPI description is fetched within 30 seconds and may hold 50 MB. See [Warehouse cost limits](https://moberghr.github.io/beacon/getting-started/configuration/#warehouse-cost-limits). Queries may only name objects in the database (catalog, project) the data source is configured for: the gate refuses other databases, remote servers, files and dynamic SQL. ## Cross-database queries A query can have several steps on different data sources. Each step's result is materialised (`@result1`, `@result2`, …) and a final query joins them in an in-memory SQLite database. Nothing is joined across engines at the database level, and the final query passes the same gate. See [Queries](https://moberghr.github.io/beacon/features/queries/). ## Monitoring your own application's database To let Beacon (and MCP clients) read your application's own EF Core model (with a table allow-list, column masking and a read-only connection string that is never stored) use `ExposeDbContext(...)` instead of creating a data source by hand. See [Extending Beacon](https://moberghr.github.io/beacon/integrate/extending/#your-dbcontext-as-a-data-source). ## CloudWatch CloudWatch data sources run Logs Insights queries, not SQL, so they cannot be queried through MCP `query` (the SQL gate has no dialect for them and refuses the statement). Beacon's own CloudWatch features use the configured log groups only, so restrict the IAM role to them. ## Notes - Connector registration also fills process-wide registries (engine → connection factory). Register connectors once, at startup. - The data-quality SQL generator supports PostgreSQL, SQL Server and MySQL. --- # AI providers > Turn on Beacon's AI layer: UseAI and AddBeaconAI, the four LLM providers and their settings, the request queue, runtime switching, local embeddings for semantic retrieval, and what data reaches the provider. Source: https://moberghr.github.io/beacon/integrate/ai/ Beacon's AI layer powers natural-language questions (`ask`), project documentation, grounding and the learning loop, and AI actors. It is optional, except that the MCP package depends on the AI package, so an MCP host always registers it. ## Turning it on ```csharp builder.Services.AddBeaconServices(builder.Configuration, options => { options.UseAI = true; // … }); builder.Services.AddBeaconAI(builder.Configuration); // binds Beacon:LLM, registers providers and the queue ``` Set both. `UseAI = false` registers stand-ins for the AI-only services; `UseAI = true` without `AddBeaconAI` leaves those services unregistered. ## Choosing a provider Configure the provider in host configuration (`Beacon:LLM`). At startup Beacon builds the provider from that section. An Admin can also change the provider in **Admin settings** (`/beacon/admin-settings`): saving swaps the active provider immediately, without a restart. If the new configuration cannot be built, no provider is active until it is fixed; the previous one is never kept. :::note The provider is built from `Beacon:LLM` each time the process starts. Keep that section in step with what you choose in Admin settings, or save the Admin settings again after a restart. ::: | `Provider` | Required | Notes | |---|---|---| | `OpenAI` | `ApiKey`, `Model` | Calls the public OpenAI API. | | `Claude` (shown as **Anthropic** in the UI) | `ApiKey`, `Model` | Calls `https://api.anthropic.com`. | | `AzureOpenAI` | `Endpoint`, `ApiKey`, `Model` (the **deployment name**) | | | `Bedrock` | `Region`, `Model` (model id) | Needs the `Moberg.Beacon.AI.Bedrock` package and `AddBeaconBedrock()`; without them a stored Bedrock setting leaves no active provider. `BedrockAuthMode`: `IamRole` (default) → the default AWS credential chain (instance role, environment); `AccessKey` → `AwsAccessKeyId` and `AwsSecretAccessKey`; `TemporaryCredentials` → those plus `SessionToken`. A legacy `ApiKey` of `accessKey:secretKey` is used when both key fields are empty; under `IamRole` a warning is logged. | ```json { "Beacon": { "LLM": { "Provider": "Claude", "ApiKey": "", "Model": "", "FastModel": "", "Limits": { "MaxConcurrentRequests": 20 } } } } ``` ```json { "Beacon": { "LLM": { "Provider": "Bedrock", "Region": "eu-central-1", "Model": "" } } } ``` `Model` and `FastModel` are free text. Use whatever model ids your provider offers. `Model` does the work (SQL generation, documentation, actors); `FastModel` names a smaller model for the short classification calls on the `ask` path (intent classification and data-source routing); every other call uses `Model`. Leave it empty, or equal to `Model`, to use `Model` for all calls; if the fast model cannot be set up or fails when called, those calls also fall back to `Model`. ### The request queue Every LLM call goes through one queue that allows at most `Limits:MaxConcurrentRequests` calls at a time (default 50). Background work (documentation generation, document indexing, learning, scheduled AI actor cycles, eval judging) holds at most half of them (at least one), so interactive calls always find room. A call waits for a slot at most `Limits:InteractiveQueueWaitSeconds` (default 60) or, for background work, `Limits:BackgroundQueueWaitSeconds` (default 300), each capped at an hour, and at most `Limits:MaxQueuedRequests` interactive calls (default 200) and half as many background calls (at least one) wait at once, so a burst of background work cannot turn interactive calls away; past either limit the call fails with `LlmQueueSaturatedException` (*The LLM is busy. Try again later.*) without reaching the provider. The wait and waiter limits are read at startup; the concurrency limit follows **Max concurrent requests** in Admin Settings as soon as it is saved (a lower limit takes effect as running calls finish and never cancels one). Each attempt holds a slot only while it runs. For Claude and Bedrock the queue is the only retry layer (the Bedrock SDK makes no retries of its own, except that a request the service refuses because the host's clock is more than four minutes off is signed again with the service's time and sent once more): a transient failure (Claude 408, 429, 5xx or 529; Bedrock throttling, service unavailable, model not ready, model timeout, internal error or any 408, 429 or 5xx; a connection failure; a provider timeout) is tried up to three times, waiting a random share of 1, 2, 4… seconds (at most 30) or, when the provider sent `Retry-After`, that wait plus up to 10 % and never less than the random share (a wait over 60 seconds fails the call); a call that still fails surfaces as `LlmTransientException`. OpenAI and Azure OpenAI retry 408, 429, 5xx and connection failures themselves (up to three retries) and the queue does not retry them again: a status they stopped retrying surfaces as an `HttpRequestException` carrying it, a connection failure or timeout as an `AiServiceException`. A cancelled call is never retried. `TokensPerMinute` and `RequestsPerMinute` apply only while `Limits:RateLimitsEnabled` (Admin Settings: **Enforce requests/tokens per minute**) is on; it is off by default. Each is then a token bucket holding a minute's allowance and refilling evenly over the minute: every attempt reserves one request and an estimate of its tokens (prompt characters / 4 plus `MaxTokens`) before it is sent, waits for them within its queue wait (or fails with `LlmQueueSaturatedException` when they cannot come in time), holds them while it waits for a slot (an attempt that is never sent gives them back), and settles the tokens with the usage the response reports when it reports the input tokens (otherwise the estimate stands). Calls wait for the buckets in arrival order, so a large call is never overtaken by smaller ones; an interactive call may pass waiting background work, and background work leaves a quarter of each bucket for interactive calls. `0` or less is unlimited. `MonthlyBudget` is recorded, not enforced; use your provider's own spend limits for a hard cap. ## Local embeddings (optional) Semantic retrieval (finding the right tables, examples and documentation chunks for a question) can use a local ONNX embedding model. Nothing leaves your process. The model support is a separate package: `dotnet add package Moberg.Beacon.AI.Embeddings.Onnx`, then call `services.AddBeaconOnnxEmbeddings()` before or after `AddBeaconAI`. Without it, `AddBeaconAI` registers a disabled service; if `Enabled` is `true` it logs one warning the first time the embedding service is used (not at startup). ```json { "Beacon": { "Embeddings": { "Enabled": true, "ModelPath": "/models/bge-small-en-v1.5/model.onnx", "TokenizerPath": "/models/bge-small-en-v1.5/vocab.txt" } } } ``` - Model: **bge-small-en-v1.5** (384 dimensions, 512 tokens). Beacon does not ship the model files; download them and mount them read-only. - Embeddings are used only when `Enabled` is `true` **and** both files exist. Otherwise Beacon falls back to keyword retrieval. - On PostgreSQL vectors live in a `pgvector` column with an HNSW index; on SQL Server they are compared in memory. - Schedule the re-index jobs (`mcp-embedding-reindex`, `mcp-docchunk-reindex`); see [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/#recurring-maintenance-jobs). ## What reaches the LLM provider By default Beacon sends **schema and documentation, not row values**: | Data | Sent? | |---|---| | Table and column names, types, relationships, documentation, glossary, verified examples | Yes (this is the grounding) | | The user's question | Yes | | Sample values from tables | Only with the per-project setting **sample-value collection** (off by default); never from PII or secret-like columns | | Values probed to ground a question (value grounding) | Only with **value grounding** on (off by default) | | Query results for an LLM judge in evaluations | Only with **eval judge** on (off by default) | | Database error text | Redacted: value-carrying errors are reduced to their class | Set the deployment lock `Beacon:Mcp:ForbidRowDataToLlm: true` to pin all three switches off for every project. Every LLM request that carries row-derived data is logged as event **7123 `RowDataSentToLlm`** with caller, project, data source ids and categories, never the values. ## Features that use AI | Feature | Needs | |---|---| | MCP `ask`, `get_query_context` | AI on; a provider for `ask` | | Project documentation | AI on, a provider, `EnqueueProjectDocumentation` in your scheduler | | Learning loop, evaluations | AI on; scheduled maintenance jobs; `EnqueueMcpEval` for eval runs | | AI actors | AI on, a provider, `EnqueueAiActorThinkCycle` in your scheduler | The MCP `query`, `dry_run`, `search`, `get_context` and `get_documentation` tools work without an LLM provider. More: [AI integration](https://moberghr.github.io/beacon/features/ai-integration/) · [Knowledge base & grounding](https://moberghr.github.io/beacon/mcp/knowledge-base/) · [AI actors](https://moberghr.github.io/beacon/features/ai-actors/). --- # Embedding saved queries > Show one approved saved query as a read-only table inside another site: turn it on, create a grant, exchange it for a short-lived token from your backend, frame the page, and what the design does and does not protect. Source: https://moberghr.github.io/beacon/integrate/embedding/ Beacon can render one approved saved query as a read-only table inside an iframe on your own site. The viewer never signs in to Beacon and never sees a Beacon credential that lasts. This is separate from [embedding Beacon in your host](https://moberghr.github.io/beacon/integrate/) (the packages) and from [MCP](https://moberghr.github.io/beacon/mcp/connect/). Embedding is **off by default** and is a deployment lock. While `Beacon:Embed:Enabled` is `false`, no embed route is mapped (each answers 404), no grant can be created and no key is needed. An [MCP gateway](https://moberghr.github.io/beacon/mcp/gateway/) host never serves the embed routes. ## How it works There are two credentials, so that a long-lived authority never reaches the browser. 1. **Grant** (stored, revocable, long-lived). A SecurityAdmin or Admin creates it in Beacon. It pins one project, one saved query and one approved version of it, the sites allowed to frame it, an expiry, and a value or a lock for every query parameter. Creating it produces a **grant secret** (`bge_...`), shown once. Only its SHA-256 is stored. 2. **Token** (stateless, short-lived). Your backend sends the grant secret to Beacon and gets an HS256 token that lives 5 minutes by default and never more than 15. Your page puts the token in the iframe URL. The browser only ever holds the token. A token buys one thing: a render of the grant's pinned query version. It opens no other route, runs no other SQL and cannot reach MCP. ## Set up ### 1. Generate a signing key and turn embedding on The signing key is separate from `Beacon:EncryptionKey`. It must be base64 of at least 32 random bytes: ```bash openssl rand -base64 32 ``` Supply it as a secret, not in a file under source control: ```bash export Beacon__Embed__Enabled=true export Beacon__Embed__SigningKey='' ``` Beacon refuses to start when the key is missing, short, plain text, a placeholder, or equal to `Beacon:EncryptionKey`, one of `Beacon:PreviousEncryptionKeys` or the obsolete `Beacon:EmbedToken` key. The message never contains the key. Changing the key invalidates every outstanding token (at most 15 minutes of them); your backend simply mints again. Every key and limit is in the [`Beacon:Embed` reference](https://moberghr.github.io/beacon/getting-started/configuration/#embedding-beaconembed). If Beacon runs behind a reverse proxy or load balancer, the host must run `UseForwardedHeaders` with the proxy's address in `KnownProxies` or `KnownNetworks`, before `UseBeaconAuthentication`, and set `Beacon:PublicBaseUrl` when TLS ends at the proxy. Without it every viewer shares the proxy's address and one rate-limit budget, and every render is refused because the viewer's `https` origin does not match Beacon's `http` one. A host `FallbackPolicy` that names `AuthenticationSchemes` also replaces the embed caller and every call answers 401: leave it without schemes or exempt the embed paths. ### 2. Make sure the query can run The query must have an approved, runnable version (the same condition as an [MCP saved-query tool](https://moberghr.github.io/beacon/mcp/)), and its project must link every data source the version reads. If a newer version is approved later, the grant stops working (`superseded`) and you create a new one: what was reviewed for embedding is what runs. ### 3. Create a grant Open **Embeds** in the sidebar (shown only to a SecurityAdmin, and only while embedding is on). Choose the project and query, then: - **Origins**: each is an exact `https://host[:port]`, with no path and no wildcard. 1 to `MaxOrigins` (default 10). `http://localhost` is accepted only in Development. - **Expiry**: at most `MaxGrantDays` (default 90). - **Parameters**: every parameter of the query is either *fixed* (a value stored on the grant) or *locked* (your backend supplies it when it asks for a token). Viewers cannot change either. An unknown or missing parameter is refused. While the approval workflow is on (and self-approval is not allowed), whoever requested or approved the query's active version cannot create a grant for it; someone else must. Copy the secret now; it is not shown again. The same operations are `POST /beacon/api/embed-grants`, `GET /beacon/api/embed-grants` and `POST /beacon/api/embed-grants/{id}/revoke`. A grant is never deleted, so the audit trail stays intact. To rotate a secret, revoke the grant and create a new one. The render acts as the **grant's creator**, with at most Viewer rights and never the Admin bypass. Creation is refused when the creator could not render the query, for example an Admin who is not a member of a restricted project. If the creator is later disabled or loses the membership, the grant renders nothing. ### 4. Exchange the secret for a token on your backend Never call this from a browser. The grant secret belongs on your server. ```bash curl -sS -X POST https://beacon.example.com/beacon/api/embed/tokens \ -H "Authorization: BeaconEmbedGrant $BEACON_EMBED_GRANT_SECRET" \ -H "Content-Type: application/json" \ -d '{ "parameters": { "customerId": "42" }, "ttlSeconds": 300 }' ``` ```json { "token": "eyJ...", "expiresAt": "2026-10-24T12:05:00Z", "embedPath": "/beacon/embed/3f9c...e1#t=eyJ..." } ``` `parameters` holds a value for each *locked* parameter, as text. `ttlSeconds` is optional. Without it the token lives `TokenTtlSeconds` (default 300). The backend may ask for anything from 1 up to `MaxTokenTtlSeconds` (default 900), so it can also ask for a longer token than the default. The token never outlives the grant. To bound token lifetime for every caller, lower `MaxTokenTtlSeconds`. The same call from a server in C#: ```csharp using var request = new HttpRequestMessage(HttpMethod.Post, "https://beacon.example.com/beacon/api/embed/tokens"); request.Headers.TryAddWithoutValidation("Authorization", $"BeaconEmbedGrant {grantSecret}"); request.Content = JsonContent.Create(new { parameters = new { customerId = "42" } }); using var response = await httpClient.SendAsync(request); response.EnsureSuccessStatusCode(); var issued = await response.Content.ReadFromJsonAsync(); // issued.EmbedPath is "/beacon/embed/{id}#t="; prefix it with Beacon's public origin. public sealed record IssuedEmbedToken(string Token, DateTimeOffset ExpiresAt, string EmbedPath); ``` Mint per page view. Do not cache a token past its expiry or share one between viewers. ### 5. Frame the page Put the origin and `embedPath` in the iframe's `src`. The token travels in the URL **fragment**, so it is never sent to a server and never appears in a `Referer` header or an access log. The embed page removes it from the address bar as soon as it loads and sends it only in an `Authorization` header. ```html ``` When the token expires, the page shows "expired, reload" and posts this message to the host page so it can mint a new token and set a new `src`. The message carries no data, only its `type`: ```js window.addEventListener('message', (event) => { if (event.origin !== 'https://beacon.example.com') return; if (event.data?.type === 'beacon-embed:expired') { // fetch a fresh embedPath from your backend and set iframe.src } }); ``` The table shows the title, the columns and rows, a note when the result was cut at `MaxRows` (default 1,000, or the project's row limit if that is lower), and an empty state. It follows the viewer's light or dark setting. ## What Beacon enforces - **One query, one version.** The render runs the pinned version's stored steps through the same read-only SQL gate and row-security resolver as every other Beacon query. Parameter values are bound as database parameters, never placed in the SQL text. - **PII is always masked.** There is no per-grant or per-project opt-out; custom patterns still apply. - **Access is checked live** on every exchange and every render. Revoking a grant, expiring it, superseding its version, disabling its creator or unlinking a data source stops every outstanding token at once. - **No ambient credentials.** The three embed routes (`POST /beacon/api/embed/tokens`, `POST /beacon/api/embed/render`, `GET /beacon/embed/{id}`) are authenticated only by the embed credential. The Beacon sign-in cookie, API keys and bearer tokens do not authenticate them, and a Beacon cookie is never created or renewed on them. The `Authorization` scheme is `BeaconEmbed`, deliberately not `Bearer`. - **Generic refusals.** Expired, badly signed, revoked, superseded and unknown all answer `401 {"error":"embed_unavailable"}`. The reason goes to the [audit trail](https://moberghr.github.io/beacon/security/audit-trail/), not to the caller. - **Frame headers.** The embed page is served with `Content-Security-Policy: default-src 'none'; script-src 'self'; style-src 'self'; connect-src 'self'; img-src 'self' data:; font-src 'self'; base-uri 'none'; form-action 'none'; frame-ancestors `, plus `Referrer-Policy: no-referrer`, `Cache-Control: no-store` and `X-Content-Type-Options: nosniff`. A grant that is unknown, revoked or expired gets 404 with `frame-ancestors 'none'`. Render responses are `no-store`. - **Frame marker.** The served page carries a marker (an HMAC of the grant's public id under the signing key) that it sends back with every render. A render without it is refused, so a token alone cannot be replayed against the render route from a script or another page. - **Same origin only.** A render whose `Origin` header names anything other than Beacon is answered 403. There is no CORS: the render is called by Beacon's own embed page, and the browser enforces the third-party list through `frame-ancestors`. - **Rate limits.** Per grant for exchanges and renders, per client address for page loads and refusals, and a cap on renders running at once. Past a limit the answer is 429 (a full render pool is 503). See the [reference](https://moberghr.github.io/beacon/getting-started/configuration/#embedding-beaconembed). - **Audit.** Grant creation and revocation, every token issued, every refusal (capped per minute) and every render are recorded without tokens, secrets, origins, parameter values or SQL. See [Embedding events](https://moberghr.github.io/beacon/security/audit-trail/). ## Threat model: what it does not protect - **The token is readable.** It is signed, not encrypted. Anyone holding it, including the viewer, can base64-decode its payload and read the grant id and the **locked parameter values**. Do not put a secret in a locked parameter. Use a value that identifies what to show (a customer id), never one that authorises something. - **Whoever holds a live token can render.** The token is the credential until it expires, so the short lifetime is the bound on theft. Lower `MaxTokenTtlSeconds` (it is the bound; `TokenTtlSeconds` is only the default) and mint per page view. - **An unframed open still renders.** Opening `embedPath` in its own tab shows the same table. `frame-ancestors` stops other sites from framing the page; it is not a check on who is looking. The token holder is the intended viewer, so Beacon does not block a top-level visit. - **Origins limit framing, not access.** The origin list is enforced by the viewer's browser. A server-side client can fetch the embed page for its own purposes, but without a token the page shows only "This view is unavailable". - **The grant secret is a password.** Anyone with it can mint tokens until the grant is revoked or expires. Store it as a secret on your backend, never ship it to a browser or commit it. If it leaks, revoke the grant at once. - **Data only as safe as the query.** The grant exposes what the pinned query returns to every viewer who can obtain a token. Review that version as you would for publishing it. - **Row-level security that cannot be applied refuses the render.** The same `Filtered` and `Denied` decisions as for MCP apply to the grant's creator. ## Troubleshooting | Symptom | Likely cause | |---|---| | Every route answers 404 | `Beacon:Embed:Enabled` is `false`, or the host is an MCP gateway | | Beacon does not start | The signing key is missing, weak or equal to another Beacon key. The startup message names the setting, not the key | | Exchange answers 401 | Wrong or revoked secret, the grant expired, or its query version was superseded. The audit trail records which | | Frame is blank and the console reports `frame-ancestors` | The embedding site's origin is not on the grant, or has a different scheme or port | | Every render answers 403 behind a proxy | `UseForwardedHeaders` is missing, or `Beacon:PublicBaseUrl` is not set where TLS ends at the proxy | | Every call answers 401 with a valid credential | A host `FallbackPolicy` names authentication schemes | | 429 for every viewer | Without forwarded headers all viewers share the proxy's address | | 503 with `Retry-After` | The render pool is full, or access could not be determined; retry | --- # Extending Beacon > Make your own application part of Beacon: expose your EF Core DbContext as a read-only data source, import your Markdown docs as grounding, turn your HTTP endpoints into MCP tools, and replace Beacon's extension points. Source: https://moberghr.github.io/beacon/integrate/extending/ An embedded Beacon can know your application, not only your databases. Three host extensions, all opt-in and all scoped to one **host project**: | Extension | You get | Page | |---|---|---| | `ExposeDbContext` | Your EF Core model as a read-only data source, with a table allow-list and column masking | [Host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/) | | `ExposeDocs` | Your Markdown documentation imported as grounding for `ask`, `search` and `get_documentation` | [Host docs](https://moberghr.github.io/beacon/mcp/host-docs/) | | `AddHostEndpointTools` | Your HTTP endpoints as MCP tools `api_`, dispatched in-process through your own authorization | [Host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/) | ## Your DbContext as a data source ```csharp builder.Services.AddBeaconServices(builder.Configuration, options => { /* … */ }) .ExposeDbContext(x => { x.ProjectName = "Shop"; // the host project; defaults to the data-source name x.ReadOnlyConnection("ShopReadOnly") // ConnectionStrings:ShopReadOnly, a read-only login .AllowTables("Orders", "OrderLines", "Customers") .AllowEntity() .ExcludeColumns(y => y.Table == "Customers" && y.Name == "Notes") .MaskColumns(y => y.Name is "Email" or "Phone"); }) .AddSqlServerConnector() .UseSqlServer(); // ends the chain: ExposeDbContext comes first ``` - Tables are **default-deny**; unknown names fail startup. - Columns whose names look like secrets (`password`, `token`, `apikey`, `hash`, `otp`, `credential`, `securitystamp`, …) and every column of credential tables (`AspNetUserTokens`, `DataProtectionKeys`, `*Keys`, `*Tokens`) are always excluded unless you lift one explicitly with `IncludeSecretLikeColumn("Table.Column")`. - The connection string is **never stored**. Beacon keeps only its name and reads it from configuration when it connects. - SQL Server and PostgreSQL models are supported. - Every statement is checked against the exposed slice in the execution gate and again in the provider; masked columns are masked by output position. ## Your docs as grounding ```csharp .ExposeDocs(x => { x.ProjectName = "Shop"; x.FromDirectory("docs", "**/*.md").Exclude("drafts/**"); // x.FromEmbeddedResources(typeof(Program).Assembly, "Shop.Docs."); // x.MaxDocumentBytes = 512 * 1024; }) ``` `.md`, `.markdown` and `.txt` files are imported into the project, chunked and embedded. MCP clients read them through `search` (`[IMPORTED_DOC]` results), `get_documentation document=…` and `get_context format=agents_md`. ## Sync at startup Both extensions need one startup call, **after** Beacon's schema is migrated: ```csharp var app = builder.Build(); Beacon.Core.ServiceConfiguration.UseBeacon(app.Services, createSchema: true); await app.Services.SyncBeaconHostAsync(); // data sources first, then docs; idempotent; replica-safe lock ``` `SyncBeaconHostDataSourcesAsync()` syncs the data sources only. ## Your endpoints as MCP tools ```csharp builder.Services.AddBeaconMcp() .AddHostEndpointTools(x => { x.ProjectName = "Shop"; // the host project the tools belong to // x.AllowWriteTools = true; // off by default }); app.MapGet("/api/orders/{id:int}", (int id, OrderService orders) => orders.Find(id)) .RequireAuthorization("ViewOrders") .WithBeaconTool("order_by_id", "One order by id.", readOnly: true); ``` Or on a controller action, or on a handler class whose framework copies class attributes into its endpoint (a Moberg Warp handler): `[BeaconTool("customer_orders", Description = "…", ReadOnly = true)]`. - `ReadOnly` must be declared explicitly; Beacon never infers it from the HTTP verb. - A read-only tool on a verb other than GET or HEAD (a POST grid or search read) is named in a startup Warning; `AllowReadOnlyPost = false` exposes read-only tools on GET and HEAD only. - Route values pass the route's constraints, and calls are capped per caller (`MaxConcurrentCallsPerCaller`, 16) and in total (`MaxConcurrentCalls`, 64); a call over a limit queues up to `QueueWait` (5 s) before it is refused. - The endpoint must carry authorization metadata (or your host has a fallback policy); otherwise startup refuses it. - Calls run in-process through your endpoint's own pipeline, as a host principal built for the MCP caller (`IMcpHostPrincipalFactory`). - Only mapped Entra callers (users or systems) of the host project can call them; API-key callers cannot. - Write tools (`ReadOnly = false`) need `AllowWriteTools`, are offered to **user** callers only and are audited as `api_:write`. - Above 20 tools (`NamedToolLimit`), clients see `search_api` + `call_api` instead of one tool per endpoint. - An argument that is the caller's own id in your application is bound from the caller with `x.BindFromCaller("userId", "")`: it leaves the tool's schema and the call carries the caller's value. See [Arguments the caller fills](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/#arguments-the-caller-fills). - An exception your endpoint throws runs your `IExceptionHandler`s, so a refusal reaches the agent as the status and problem a browser would get. ## Saved queries as tools without code A SecurityAdmin can publish an approved saved query as an MCP tool `q_` from the query's page. See [Saved query tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/). ## Replaceable services Beacon registers most services with `TryAdd`, so **your registration wins if you register first** (before `AddBeaconServices`, `AddBeaconApiServices` or `AddHostEndpointTools`, depending on who registers it). | Interface | Default | Replace it to | |---|---|---| | `IBeaconScheduler` | None (required) | Use your job runner. | | `IEmailAdapter` | none | Send email (`AddEmailAdapter()`). | | `IBeaconAuthenticationProvider` | user store | Check passwords somewhere else. | | `IBeaconAuthorizationProvider` | user store / allow-all | Cap users' global read/write from your own model. | | `IProjectMembershipSource` | Beacon's membership table | Take project membership from your application. Beacon's membership endpoints then answer 409. | | `IMcpCallerMapper` | `Beacon:Mcp:Callers` | Map JWT callers to projects and scopes your own way. | | `IMcpHostUserResolver` | none | Let your own users decide who may use MCP: register `HostUserMcpCallerMapper` and `HostUserMcpHostPrincipalFactory` with it (the MCP gateway does), and a delegated Entra token is a caller when your resolver knows its person; a tool call runs as the user it returns. | | `IMcpHostPrincipalFactory` | copies name/email/oid; system callers get `HostClaims` | Build the principal your endpoints expect for MCP calls. | | `IHostEndpointDispatchMiddleware` | none | Run your own middleware logic (tenant, culture, audit) around dispatched tool calls. Ships in `Moberg.Beacon.MCP` (namespace `Beacon.Core.HostEndpoints`). | | `IApprovalNotifier` | SignalR | Notify approvers elsewhere. | | `IBeaconUserContext` | `HttpContextUserContext` (Beacon.Api) | Resolve the current user differently. Replace it in both places it is registered. | | `IBeaconRequestPrincipal` | Core: no request; Beacon.Api and Beacon.MCP: the current `HttpContext` | Tell Beacon's core who the request's caller is from your own request model. Call `UseRequestPrincipal()` before or after Beacon's registrations: it replaces Beacon's own principals (Core's default and the `HttpContext` one) in any order, and Beacon's web registrations after it keep yours. A principal registered with `AddSingleton` before Beacon's registrations stays, and after them wins as the last registration. A `TryAdd*` after `AddBeaconServices` is a no-op, because Core has already registered its default. A host that maps Beacon's endpoints over Core's default fails to start. | `IProjectAccessEvaluator` is replaceable in principle but it is the single decider for project access; replace it only with great care, and keep the project-authorization tests green if you fork it. --- # Production checklist > What to set before Beacon faces real users and real data: secrets, Data Protection, reverse proxy, sign-in, locks, audit, MCP, scaling, observability and upgrades. Source: https://moberghr.github.io/beacon/integrate/production/ Go through this list before the first real user signs in. Each line links to where it is explained. ## Secrets and keys - [ ] `Beacon:EncryptionKey` comes from a secret store or environment variable, never a file in source control; its backup is stored separately from database backups. → [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/) - [ ] You know how to **rotate** it: move the old key to `Beacon:PreviousEncryptionKeys:0`, set the new one, set `Beacon:Mcp:CallerHashKey` to the old key first if you want MCP caller hashes to stay comparable, run the re-encryption job, then remove the old key once it reports nothing left under it: pending changes of recipients, data-source connections and the LLM connection made under the old key can keep it needed until they are decided. Set `Beacon:Audit:IntegrityKey` before the first rotation: while it is unset, security audit events are signed under a key derived from the caller hash key or the encryption key, and verify as *unknown key* once that secret leaves the configuration. → [Keys and rotation](https://moberghr.github.io/beacon/security/audit-trail/#keys-and-rotation) - [ ] `Beacon:DataProtection:ApplicationName` is set (one value per environment), and the key ring is **encrypted at rest** (`ProtectKeysWithCertificate` / `ProtectKeysWithAzureKeyVault`), otherwise the host does not start outside `Development`. → [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/#data-protection) - [ ] OIDC client secrets, LLM API keys and SMTP passwords come from the secret store. - [ ] `Beacon:UserManagement:SetupToken` is set for multi-instance first runs, and removed afterwards. ## Reverse proxy - [ ] TLS terminates at the proxy, and the host runs `UseForwardedHeaders` with **only your proxies** listed, first in the pipeline. The per-address request limits (`Beacon:RateLimits`: login 10 / 60 s, anonymous refusals 30 / 60 s) and the cookie's Secure flag depend on it. ```csharp var forwarded = new ForwardedHeadersOptions { ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto }; forwarded.KnownProxies.Clear(); forwarded.KnownNetworks.Clear(); forwarded.KnownProxies.Add(IPAddress.Parse("10.0.0.5")); app.UseForwardedHeaders(forwarded); ``` - [ ] `Beacon:PublicBaseUrl` is the public origin (e.g. `https://beacon.example.com`), so MCP discovery documents do not echo the client's `Host` header. → [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/#public-url-and-discovery) - [ ] `options.BaseUrl` (or **Admin settings → Base URL**) is the public UI URL **including `/beacon`**, so notification links work. - [ ] HTTPS redirection applies to `/beacon/mcp` too, because API keys travel in headers. - [ ] HSTS is sent by the host (`app.UseHsts()`) or the proxy, and `AllowedHosts` lists your real host names. - [ ] The origin check knows your public origin (`Beacon:PublicBaseUrl` or `Beacon:Security:AllowedOrigins`); after a release without its `LogOnly` warnings, `Beacon:Security:OriginGuard` is `Enforce`. → [Security headers and the origin check](https://moberghr.github.io/beacon/integrate/host-setup/#security-headers-and-the-origin-check) - [ ] Proxy and Kestrel timeouts allow long requests (AI documentation, `ask`): the reference host uses 5 minutes. ## Sign-in and access - [ ] Authorization is on (`options.Authorization.Enabled = true`). - [ ] SSO admission is narrow: `AllowedTenants`, `BlockGuests` (default on), `RequiredGroups` or `RequiredRoles`; `RequireMfa` if your provider issues `amr`/`acr`. → [Sign-in & identity](https://moberghr.github.io/beacon/integrate/authentication/#oidc--sso) - [ ] Local passwords off once SSO works, with one break-glass account in a vault; alert on event 7119 `BreakGlassSignIn`. - [ ] `Beacon:Authorization:RequireProjectMembership: true`; projects planned; data sources linked only where needed. → [Access control](https://moberghr.github.io/beacon/integrate/access-control/) - [ ] Few Admins; duties given as functional permissions; `SeparationOfDuties` on if your auditors expect it. - [ ] Approval workflow on with at least two Approvers, `RequireApprovedVersionToRun` for regulated data. → [Approvals](https://moberghr.github.io/beacon/security/approvals/) - [ ] API keys: decide `Beacon:ApiKeys:MaxLifetimeDays` and `MaxOwnerInactivityDays`. ## Data and SQL - [ ] Every data source uses a **read-only login**, including PostgreSQL and MySQL. → [Connectors](https://moberghr.github.io/beacon/integrate/connectors/#creating-read-only-logins) - [ ] Deployment locks reviewed: `Beacon:Mcp:ForceReadOnly` and `ForcePiiDetection` stay `true`; consider `ForbidRowDataToLlm` and `ForceNoContentRetention`. → [Access control](https://moberghr.github.io/beacon/integrate/access-control/#5-deployment-locks) - [ ] `Beacon:Query:MaxExecutionRows` (default 100,000) and `StatementTimeoutSeconds` (default 120) fit your workloads. - [ ] `Beacon:Mcp:Ceilings:MaxRowLimit` set if project admins must not raise the MCP row limit. - [ ] `Beacon:Migrations:Enabled` stays `false` unless you need data migration. It is the only write path. - [ ] Notification destinations restricted as needed: `Beacon:Notifications:AllowedHosts:*`, `DisabledTypes`; private networks stay unreachable unless listed in `AllowedPrivateNetworks`. → [Notifications](https://moberghr.github.io/beacon/features/notifications/) ## Audit and observability - [ ] Data retention reviewed: `Beacon:Retention:Days` set (nothing is purged while it is `0`, the default; 90 recommended) and its targets match your policy, `ContentDays` set (30 recommended for regulated data), `Locked` on where no data may be kept forever, and `IDataRetentionService.RunAsync` scheduled daily. → [Data retention](https://moberghr.github.io/beacon/getting-started/configuration/#data-retention-beaconretention) - [ ] `Beacon:Mcp:Audit:RetentionDays` set to your retention policy (kept forever while unset; 365 recommended) and the purge job scheduled; `Audit:Required: true` if a tool result must never be returned without its audit row. → [MCP observability & audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/) - [ ] Security log events (structured log events 7101 to 7123: membership, roles, sessions, break-glass, access refused, row data to LLM) are shipped to your log pipeline and alerted on. The log events themselves are not stored in Beacon's database; the actions most of them report are also recorded in the security audit trail. → [Security model](https://moberghr.github.io/beacon/security/#12-audit-and-security-events) - [ ] Security audit trail: `Beacon:Audit:IntegrityKey` set from a secret store (before you change the encryption key or the caller hash key), the verification job (`ISecurityAuditVerificationService.VerifyNewEventsAsync`, hourly) and the retention purge (`ISecurityAuditRetentionService.PurgeExpiredAsync`) scheduled, `Beacon:Audit:RetentionDays` set to your retention policy (at least 365; events are kept forever while it is unset), the `Beacon.Audit` events 9104 to 9114 shipped to your SIEM, and alerts on any increment of `beacon.security_audit.write_failures` and `beacon.security_audit.verification_failures`. → [Security audit trail](https://moberghr.github.io/beacon/security/audit-trail/) - [ ] To make the trail append-only at the database as well, Beacon's connection uses a role that cannot update or delete `security_audit_events` rows other than through the purge, or triggers do the same. → [What the trail does and does not prove](https://moberghr.github.io/beacon/security/audit-trail/#what-the-trail-does-and-does-not-prove) - [ ] Where MCP audit content is retained, the SQL of every statement a call ran is stored with it: reviewed against your privacy policy; `Beacon:Mcp:Audit:RequireContent` pins retention on if the evidence must be kept. → [MCP observability & audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/#keeping-audit-content-the-requirecontent-lock) - [ ] Structured JSON console logs (`Logging:Console:FormatterName=json`, `IncludeScopes=true`) if your platform correlates by trace id. - [ ] OpenTelemetry: add `AddBeaconInstrumentation()` to your tracer and meter providers, and `AddBeaconMcpSdkTracing()` to the tracer provider for the MCP SDK's `tools/call` spans (their status descriptions cleared); you choose the exporter. Keep notification HTTP calls out of traces (their URLs carry secrets), and the REST API connector's (their paths and queries can carry identifiers or a credential). `NotificationRequests.IsNotificationRequest` and `ApiConnectorRequests.IsApiConnectorRequest` identify them. - [ ] Health check `GET /beacon/api/health` (anonymous, checks the database within 5 s) wired to your load balancer. It answers before the first administrator exists too, so a brand-new deployment passes the probe while setup is still pending. The route is limited to 120 requests per minute per remote IP (429 with `Retry-After`), shares one database check for two seconds, and names no cause in a 503 to an anonymous caller. Behind a proxy, run `UseForwardedHeaders` so the limit is per client, not per proxy. - [ ] The OpenAPI document needs a signed-in Beacon caller outside Development when mapped with `MapBeaconOpenApi()`. List `/openapi` in `Beacon:Authentication:AdditionalPaths` so the Beacon session reaches it, or set `Beacon:OpenApi:Anonymous` to `true` to publish it. - [ ] Regular expressions: every pattern Beacon runs on user, catalog or model text carries its own match timeout (250 ms to 1 s), so no setting is needed. For your own host code, you can add a process-wide backstop with `AppContext.SetData("REGEX_DEFAULT_MATCH_TIMEOUT", TimeSpan.FromSeconds(2))` (or the same name under `configProperties` in `runtimeconfig.json`) before the first regex is built; it applies only to patterns that set no timeout of their own. ## Scaling - [ ] Several instances: shared Data Protection key ring (Beacon's database does this), a SignalR backplane (`ConfigureSignalR`), **session affinity for `/beacon/mcp`** (legacy MCP clients hold an `Mcp-Session-Id` in memory), and a job runner that fires each recurring job once. - [ ] Settings saved on one instance (admin settings, MCP settings) reach the others within about a minute: each instance checks the settings history for a newer change at most once every 60 seconds, on its next settings read, then drops its cached copies and applies the stored base URL and LLM configuration again. The instance that saved applies the change at once and, because its own history rows move the stamp, applies the stored settings once more within the minute (harmless: it applies what is stored). A re-apply that fails is tried again at the next check. - [ ] Beacon shares the host's `IMemoryCache`: its keys start with `beacon:` and every entry has a size of 1, so a host `MemoryCacheOptions.SizeLimit` counts each Beacon entry as one. - [ ] Migrations run once per deployment, not from every replica at the same time. → [Metadata database](https://moberghr.github.io/beacon/integrate/database/#creating-and-migrating-the-schema) - [ ] Database pool sized for job workers plus web traffic. ## Upgrades - [ ] All `Moberg.Beacon.*` packages on the same version. - [ ] Read the release notes of every version you upgrade to: a release can change defaults, and the notes say what to do. - [ ] Back up the metadata database before migrating; migrations are append-only and run forward. - [ ] After upgrading every instance, run one-off jobs the release notes name (for example recipient-secret encryption). --- # Troubleshooting > Startup failures, empty pages, 401s and 404s, jobs that do nothing, MCP clients that cannot connect: symptoms, causes and fixes for an embedded Beacon. Source: https://moberghr.github.io/beacon/integrate/troubleshooting/ ## The host does not start | Message (or symptom) | Cause | Fix | |---|---|---| | `Implementation of IBeaconScheduler is required.` | No scheduler registered. | `options.AddBeaconScheduler()`. → [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/) | | `CS9137` (interceptors) at build | `AddOpenApi`/`MapOpenApi` without a direct `Microsoft.AspNetCore.OpenApi` reference. | Add the package to your project, or drop the two calls. | | `CS0122 'RoleService' is inaccessible` | A claims transformation copied from the reference host uses Beacon's internal role constants. | Use the version on [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/#the-claims-transformation). | | An `InvalidOperationException` about `Beacon:EncryptionKey` | Key missing, too short, not random, a published example, or a `${…}` placeholder. | `openssl rand -base64 32` into a secret store / `Beacon__EncryptionKey`. | | `Connection string 'BeaconContext' is not configured` | `UsePostgreSql()`/`UseSqlServer()` without `ConnectionStrings:BeaconContext`. | Set it, or pass a connection string. In development check that user-secrets load (they only load in `Development`). | | An error about `Beacon:Schema` | The key is set to something other than `beacon`. | Remove it. The schema is fixed. | | `Beacon:DataProtection:ApplicationName must be set outside Development` | `AddBeaconDataProtection` outside `Development` without an application name. | Set it, or run with `ASPNETCORE_ENVIRONMENT=Development` locally. | | Data Protection keys "persisted without encryption at rest" | Outside `Development` the key ring has no protector. | Chain `ProtectKeysWithCertificate(...)` / `ProtectKeysWithAzureKeyVault(...)`, or set `Beacon:DataProtection:AllowUnencryptedKeys` deliberately. | | `Beacon:UserManagement:SetupToken must be at least 32 characters` | Short setup token. | Use a longer one or unset it. | | Local login off and "nobody could sign in" | `LocalLogin:Enabled=false` without SSO, external login or a break-glass name. | Enable OIDC, configure `ExternalLogin`, or add a break-glass name. | | OIDC: missing Authority/ClientId/ClientSecret or admission rule | `Oidc:Enabled` without required values or without `AllowedTenants` (or `AllowAnyTenant`). | Complete the section. | | `Beacon:Mcp:Callers` validation errors | Callers configured without `AllowedTenants`, without bearer JWT validation, or with both `ClientId` and `ObjectId`. | See [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/#option-b-entra-id-callers). | | `Beacon host endpoint tools are misconfigured:` | A `[BeaconTool]` endpoint without `ReadOnly`, without authorization metadata, with an untrusted auth scheme, or a write tool without `AllowWriteTools`. | Fix each listed endpoint. | | `AddHostEndpointTools can only be called once.` | Called twice. | Call it once, after `AddBeaconMcp()`. | | `The MCP tool name '…' is claimed by both …` | The saved-query tools and the host endpoint tools (or another tool source) serve the same name or name prefix. | Rename the endpoint tool. (An attribute tool with such a name only logs a warning; calls to it reach the attribute tool.) | | Options validation for `Beacon:Query`, `Beacon:Mcp`, `Beacon:ApiKeys`, `Beacon:Notifications`, `Beacon:Authorization`… | A value is out of range or malformed. | The message names the key; see [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/). | ## Database | Symptom | Cause | Fix | |---|---|---| | `3D000: database "…" does not exist` | The database itself is missing; `UseBeacon` creates the schema, not the database. | `CREATE DATABASE beacon;` (or `POSTGRES_DB=beacon` on the container). | | `relation "…" does not exist` / `Invalid object name 'beacon.…'` | Beacon's schema was never migrated. | Call `Beacon.Core.ServiceConfiguration.UseBeacon(app.Services, createSchema: true)` at startup. → [Metadata database](https://moberghr.github.io/beacon/integrate/database/) | | `extension "vector" is not available` | pgvector is not installed for this PostgreSQL server. | Install pgvector (Azure: allow `VECTOR` in `azure.extensions`), or use an image that has it (e.g. `pgvector/pgvector`). | | `permission denied to create extension "vector"` | The migrating login cannot create extensions. | Create it once as an administrator: `CREATE EXTENSION vector;`. | | `type "vector" does not exist` with your own `NpgsqlDataSource` | The extension lives in a schema that is not on the search path. | Install it in `public` (Beacon's search path is `beacon, public`). | ## The UI | Symptom | Cause | Fix | |---|---|---| | 404 at `/` | The UI lives at `/beacon`. | Open `/beacon`, or redirect `/` to it yourself. | | 404 for `/beacon` or blank page; asset 404s | `MapBeaconUi()` or `UseStaticFiles()` missing; or, building from source, the React app was not built (`Debug` builds skip it). | Add both calls; build `Release`, or `-p:BuildReactInDebug=true`. | | Endless redirect to `/login` (not `/beacon/login`) | `AddBeaconCookieAuthentication("/")`. | Pass `"/beacon"`. | | Always redirected to `/beacon/setup` (the health check and the MCP discovery documents still answer) | User management on and no administrator yet. | Complete first-run setup with the setup token (printed once to the console, or `Beacon:UserManagement:SetupToken`). | | Signed in, but every API call answers 401 | Middleware order: `UseBeaconAuthentication` must run before `UseAuthorization`. | Follow the [pipeline order](https://moberghr.github.io/beacon/integrate/host-setup/#the-middleware-pipeline). | | Saves fail with 400 "antiforgery" | No `AddAntiforgery`/`UseAntiforgery`, or a different header name without the SPA header. | Configure antiforgery with `HeaderName = "X-XSRF-TOKEN"`. | | An error shows as "Internal server error" in the UI | Business-rule exceptions are not mapped to problem+json. | Add the [error handler](https://moberghr.github.io/beacon/integrate/host-setup/#error-responses). | | A project, query or data source is missing for a user | Project access: the project is restricted and they are not a member, or the data source is not linked to a project they can read. | Add them as a member or link the data source. Restricted resources answer 404 on purpose. | | No live updates | Realtime off, or `AddBeaconApiServices()` missing, or a proxy blocks WebSockets on `/beacon/api/hub`. | Check `realtimeEnabled` in `GET /beacon/api/auth/me`. | ## Scheduled checks | Symptom | Cause | Fix | |---|---|---| | Subscriptions never run | Your scheduler does not register the recurring job, or no worker is running. | Check `AddOrUpdate` and your runner's dashboard. | | Jobs run but do nothing; log warns about a scope without project access | The job's scope was not marked as system work. | Call `ISystemWorkScope.MarkAsSystemWork()` first in every job. | | Run recorded as **Skipped** | The subscription's owner can no longer run it (removed from the project, role lowered, disabled). | Give it a new owner (UserAdmin or Admin) or restore access. | | Run refused: "not approved" | Approval workflow on and the query's current version is not approved. | An Approver approves the change. | | `NotImplementedException: … not implemented in this host` | The feature calls a scheduler member you left at its default. | Implement `AddOrUpdateDataQualityJob`, `EnqueueProjectDocumentation`, `EnqueueAiActorThinkCycle` or `EnqueueMcpEval`. | | Notification links 404 | `BaseUrl` lacks `/beacon`. | Set it to `https://host/beacon`. | | A recipient receives nothing | Destination not allowed by `Beacon:Notifications`, or awaiting approval. | Check the startup policy log line and **Approvals**. | ## MCP | Symptom | Cause | Fix | |---|---|---| | 401 | No or invalid key/token. | Send `Authorization: Bearer sk-sem_…`. | | 403 | Read-scoped key, an unmapped JWT caller, or a signed-in browser session. | Use an Execute key or map the caller; browsers use the MCP playground. | | "No projects are available to you." | The key names no reachable project. | Create the key with a project. | | "Project access could not be determined; retry the call." | A transient failure while resolving access. | Retry; check the database. | | Discovery documents show the wrong host | `Beacon:PublicBaseUrl` unset behind a proxy. | Set it. | | Calls fail randomly across instances | Legacy MCP sessions without affinity. | Enable session affinity for `/beacon/mcp`. | Still stuck? Compare your `Program.cs` with `src/Beacon.SampleProject/Program.cs`, and [open an issue](https://github.com/moberghr/beacon/issues). --- # Features > A map of every Beacon feature page: monitoring and moving data, the AI layer, and where the MCP and Security sections live. Source: https://moberghr.github.io/beacon/features/ Every feature page, grouped as in the sidebar. New to Beacon? Read [Core concepts](https://moberghr.github.io/beacon/getting-started/concepts/) first, then the [Quick start](https://moberghr.github.io/beacon/getting-started/quick-start/). ## Monitor & move data | Page | What it covers | |---|---| | [Data sources](https://moberghr.github.io/beacon/features/data-sources/) | Nine connectors (PostgreSQL, SQL Server, MySQL, Azure Synapse, Snowflake, BigQuery, Databricks, AWS CloudWatch, REST API), encrypted connections, metadata, and per-engine read-only guarantees. | | [Queries](https://moberghr.github.io/beacon/features/queries/) | Saved, versioned SQL: steps across data sources, bound `{name}` parameters, an in-memory final query, and optional change review. | | [Subscriptions](https://moberghr.github.io/beacon/features/subscriptions/) | Run a query on a cron schedule as its owner, decide which runs notify, and choose what each notification carries. | | [Notifications](https://moberghr.github.io/beacon/features/notifications/) | Five channels (Email, Teams, Slack, Jira and Webhook) to vetted recipients, with a delivery history. | | [Control Tower](https://moberghr.github.io/beacon/features/control-tower/) | Health across every subscription: success rates, health buckets, open tasks and anomaly sparklines. | | [Tasks](https://moberghr.github.io/beacon/features/tasks/) | Work items a subscription opens while its query returns rows, resolved automatically when the data recovers. | | [Anomaly detection](https://moberghr.github.io/beacon/features/anomaly-detection/) | Notify only when a run's row count is unusual against its own history. | | [Data quality](https://moberghr.github.io/beacon/features/data-quality/) | Data contracts with rule-based checks, a quality score and alerts, on PostgreSQL, SQL Server and MySQL. | | [Data migration](https://moberghr.github.io/beacon/features/data-migration/) | Copy rows from a read source into a database an Admin marked as a write target. Off unless the host enables it. | ## AI & MCP MCP has its own section: start at the [MCP overview](https://moberghr.github.io/beacon/mcp/). Its pages: | Page | What it covers | |---|---| | [MCP server & tools](https://moberghr.github.io/beacon/mcp/server/) | The governed MCP endpoint at `/beacon/mcp`: eight built-in tools, project scoping, read-only SQL and PII masking. | | [Connect AI clients](https://moberghr.github.io/beacon/mcp/connect/) | Wiring, discovery, API keys or Entra callers, and Claude Code, Claude, Cursor and VS Code set-up. | | [MCP gateway](https://moberghr.github.io/beacon/mcp/gateway/) | Your own application as an MCP server: your users, your endpoints as tools, your docs as grounding. | | [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/) | Calling MCP with Microsoft Entra ID tokens, on behalf of a user or as a system identity. | | [Saved query tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/) | Approved saved queries exposed as named, read-only MCP tools. | | [Host endpoint tools](https://moberghr.github.io/beacon/mcp/host-endpoint-tools/) | Selected host endpoints exposed as MCP tools, run in-process as the caller. | | [Host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/) | The host's own EF Core model as a read-only data source with an allow-list and masking. | | [Host docs](https://moberghr.github.io/beacon/mcp/host-docs/) | The host's markdown documentation imported into a project and searchable over MCP. | | [Knowledge base & grounding](https://moberghr.github.io/beacon/mcp/knowledge-base/) | What Beacon gives the model before it writes SQL: schema, relationships, glossary, golden examples, learned patterns. | | [AI integration](https://moberghr.github.io/beacon/features/ai-integration/) | Experimental: generated data source documentation through a swappable LLM provider. | | [AI actors](https://moberghr.github.io/beacon/features/ai-actors/) | Experimental: LLM-driven monitoring agents whose plans a person approves before anything runs. | | [MCP observability & audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/) | The audit row behind every MCP tool call, trace correlation and OpenTelemetry export. | ## Govern & secure Security has its own section: start at the [security model](https://moberghr.github.io/beacon/security/). Its pages: | Page | What it covers | |---|---| | [Security model](https://moberghr.github.io/beacon/security/) | The trust boundaries, deployment locks and guarantees in one place. | | [Authorization](https://moberghr.github.io/beacon/security/authorization/) | Projects as the access boundary, project roles and actions, functional permissions, and host seams. | | [Approvals](https://moberghr.github.io/beacon/security/approvals/) | The opt-in four-eyes review of query changes, migration jobs, Custom SQL rules, recipient destinations, data-source connections and the LLM connection. | | [User management](https://moberghr.github.io/beacon/security/user-management/) | Users, roles, password and SSO sign-in, and the first-run setup. | | [API keys](https://moberghr.github.io/beacon/security/api-keys/) | Read and Execute keys bound to their owner, with project lists and a mandatory expiry, stored hashed. | | [Security audit trail](https://moberghr.github.io/beacon/security/audit-trail/) | An append-only, signed record of sign-ins and of changes to identities, keys, access, data sources, delivery, settings, approvals and curation, read and verified by an Auditor and copied to the `Beacon.Audit` log stream. | | [Admin settings](https://moberghr.github.io/beacon/security/admin-settings/) | Runtime settings: the LLM provider, the base URL, and their change history. | For configuration keys see [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/); to embed Beacon in your own application see [Integrate Beacon](https://moberghr.github.io/beacon/integrate/). --- # Data Sources > Connect the databases and services Beacon reads: nine connectors, credentials encrypted at rest, access through projects, and per-engine read-only guarantees. Source: https://moberghr.github.io/beacon/features/data-sources/ ![Data sources](https://moberghr.github.io/beacon/img/screenshots/data-sources-light.png) A data source is one connection Beacon reads: a database, a warehouse, a CloudWatch log set or an HTTP API. Queries, subscriptions, data contracts, MCP tools and the AI layer all reach data through data sources, and data sources are reached through the [projects](#projects-and-access) that link them. ## Connectors | Kind in the UI | Engine | What Beacon reads | |---|---|---| | **Database** | PostgreSQL, SQL Server, MySQL, Azure Synapse, Snowflake | SQL, plus tables and columns for metadata | | **AWS CloudWatch** | n/a | Log groups; metadata discovers the log fields | | **Databricks** | n/a | SQL against a SQL warehouse | | **BigQuery** | n/a | SQL against a project | | **API (OpenAPI)** | n/a | An HTTP API described by an OpenAPI spec; metadata imports its endpoints | Each connector is a separate package the host registers (see [Data-source connectors](https://moberghr.github.io/beacon/integrate/connectors/)). Connection details are encrypted at rest with AES-256 under the required `Beacon:EncryptionKey` and decrypted only when Beacon connects (see [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/#encryption-key-configuration-required)). ## Create a data source Creating, testing a connection for and deleting a data source are a **DataSourceAdmin**'s (an Admin holds the permission). See [Functional permissions](https://moberghr.github.io/beacon/security/authorization/#functional-permissions). 1. Open **Data Sources** (`/beacon/data-sources`) and click **New data source**. 2. **Type.** Pick the **Data source type**. 3. **Connection.** Enter a **Name** and the connection details for that type (below). 4. **Test & save.** Review the summary, pick the **Projects** to link the data source into, click **Test connection** (it shows **Connected** or **Failed** with the reason), then **Create data source**. Without a project, the new data source is usable by Admins only until a project links it. Picking projects needs a Viewer or Editor role, and linking into a project needs that project's admin role, as [linking](#projects-and-access) does. ### Connection fields | Type | Fields (required in bold) | |---|---| | Database | **Database engine**, **Connection string**, and the [metadata options](#metadata-loading-options) | | AWS CloudWatch | **AWS region**, **Log groups (comma-separated)**, Access key id, Secret access key, AWS profile (an AWS profile, or no access key, needs `AllowHostCredentials`: see [Driver options and hosts](#driver-options-and-hosts)) | | Databricks | **Host**, **HTTP path**, **Personal access token**, Catalog, Schema | | BigQuery | **Project id**, Dataset id, Location, **Service account JSON** (a service account key) | | API (OpenAPI) | **Base URL**, **OpenAPI spec URL**, Authentication (None, API key, Bearer token, Basic auth), Include path patterns, Exclude path patterns | Connection string examples for the database engines: ```text # PostgreSQL Host=prod-db.example.com;Database=appdb;Username=beacon_readonly;Password=...;SSL Mode=VerifyFull # SQL Server / Azure Synapse Server=sql.example.com;Database=appdb;User Id=beacon_readonly;Password=...;Encrypt=Strict # MySQL Server=mysql.example.com;Database=appdb;Uid=beacon_readonly;Pwd=...;SslMode=VerifyFull # Snowflake account=myorg-myaccount;user=beacon_readonly;password=...;db=APPDB;warehouse=BEACON_WH ``` Each example authenticates the server over TLS, which Beacon requires: see [Transport security (TLS)](#transport-security-tls). :::note[Name the database] Set the database in the connection: `Database` (or `Initial Catalog`) on SQL Server, Azure Synapse, PostgreSQL and MySQL, `db` on Snowflake, the **Catalog** on Databricks, the **Project id** on BigQuery. SQL against the data source may name only that database (catalog, project) in a qualifier such as `database.schema.table`; without one configured, no database qualifier is accepted. See [Database names in SQL](https://moberghr.github.io/beacon/security/authorization/#database-names-in-sql). ::: If **Test connection** fails, check that the host can reach the server (firewall, VPN, private endpoint), that the credentials and database name are right, and that the login can read. ### API (OpenAPI) requests An API data source's credentials go only to the API it names: - **API key location.** Send the API key in a header. A new or changed configuration with the key in the query string (`ApiKeyLocation: Query`) is refused: a URL is written to server and proxy logs, and an API that echoes its URL hands the key back. A stored one keeps working, with one Warning per data source per process; change it to a header. - **Masked answers.** The data source's bearer token, API key, Basic password and Basic header value are replaced with `***` in what the API answers, before it becomes rows, also URL- or JSON-escaped. - **No redirects.** Beacon does not follow redirects: a `3xx` answer fails the query or the test and says so. Point the **Base URL** and the **OpenAPI spec URL** at their final addresses. - **Paths stay under the base URL.** A query's path (after its path parameters are filled in) may not contain `.` or `..` segments, also percent-encoded or behind a backslash, nor a `#` or a control character, and the request URL keeps the base URL's scheme, host, port and path. - **Headers.** A query may add its own headers, but not `Host`, `Forwarded`, `X-Forwarded-*`, `Via`, `X-Real-IP`, `X-Original-URL`, `X-Rewrite-URL`, the method overrides (`X-HTTP-Method`, `X-HTTP-Method-Override`, `X-Method-Override`), `Content-Length`, `Transfer-Encoding`, `Connection`, `Upgrade`, `Expect`, `Authorization`, `Cookie`, `Proxy-*`, nor the API key's own header. A header name must be an HTTP token and a value carries no control character other than a tab. - **Test connection** fetches the OpenAPI spec URL with the credentials only when it has the base URL's scheme, host and port; a description served elsewhere is fetched without them. - **Logs and traces.** The connector's HTTP client writes no request log, and neither the spec URL nor a query's path is logged. Its requests are marked: leave them out of HTTP tracing with `ApiConnectorRequests.IsApiConnectorRequest` (`Beacon.Connector.Api.Services`), as the sample host does. ### Transport security (TLS) Every data-source connection must encrypt its traffic and authenticate the server: the server's certificate must chain to a trusted certificate authority and name the host Beacon connects to. Otherwise anyone on the network path can read the credentials and the rows. Each connector checks its connection with its driver's own parser, so option synonyms and case do not matter. | Engine | Required | Server certificate from a private CA | |---|---|---| | PostgreSQL | `SSL Mode=VerifyFull`. `Require` and `VerifyCA` do not check the host name, and the default (`Prefer`) may not encrypt at all. | `Root Certificate=/path/to/ca.pem` | | SQL Server, Azure Synapse | `Encrypt` set to `Mandatory` (`True`, the default) or `Strict`, and `TrustServerCertificate` left out or `false`. | Install the CA on the Beacon host, or pin it: `HostNameInCertificate=` when it differs from the server name, `ServerCertificate=/path/to/server.cer` to accept exactly that certificate. | | MySQL | `SslMode=VerifyFull`. `Required` and `VerifyCA` do not check the host name, and the default (`Preferred`) may not encrypt at all. | `SslCa=/path/to/ca.pem` | | Snowflake | Nothing to set. `insecureMode=true`, a `certRevocationCheckMode` other than `ENABLED` and a `scheme` other than `https` are refused. | Not applicable | | API (OpenAPI) | **Base URL** and **OpenAPI spec URL** are absolute `https://` URLs. | Install the CA on the Beacon host. | | Databricks, BigQuery, AWS CloudWatch | Nothing to set: their clients always connect over HTTPS and check the certificate. | Not applicable | A CA file is read on the Beacon host, so it must exist there. A connection Beacon cannot read, or whose type has no connector registered, is never verified. How Beacon applies the rule: - **Creating a data source, changing its connection, and Test connection** refuse a connection that does not meet it. The message names the option to change, never a value from the connection string. - **A data source stored before the rule** keeps working, so an upgrade does not stop running queries and subscriptions. The data-source list reports it (`transportVerified: false`), and the log carries one Warning per data source per process, naming it by id. Replace its connection with one that meets the rule. A host that sets `Beacon:DataSources:Transport:EnforceOnExisting` to `true` refuses such a source every time it connects. - **Host-managed data sources** follow the same rules, applied to the connection string the host configures, which is read each time the source connects. That string is never refused as a new connection, since no Admin enters it in Beacon: it is checked each time the source connects, with the Warning (or, under `EnforceOnExisting`, the refusal) of a stored source. A host that changes the string to one that does not meet the rule therefore gets the Warning, not a refusal, unless it sets `EnforceOnExisting`. - **A migration write target** has no badge: Beacon never decrypts its connection to show one. The migration writer applies the same rule each time it connects to it. - **`Beacon:DataSources:Transport:AllowInsecure`** lifts the rule for development hosts whose local databases have no TLS (the sample host sets it in its Development configuration). Outside Development the host logs a Warning at startup while it is set. See [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/#data-source-transport-security). ### Driver options and hosts A connection carries the data source's own credentials and nothing that acts on the Beacon host. Each connector reads the connection with its driver's own parser and refuses these options: | Engine | Refused | Signs in as the Beacon host (lifted by `AllowHostCredentials`) | |---|---|---| | PostgreSQL | `Options`, `Passfile`, `Include Error Detail=true`, `Log Parameters=true` | A Unix-socket `Host`, `GSS Encryption Mode=Require`, a `Require Auth` that admits `GSS` or `SSPI`, no `Password` and no `SSL Certificate` (the host's `PGPASSWORD` or pgpass file) | | SQL Server, Azure Synapse | `AttachDBFilename`, `User Instance=true`, `Authentication=Active Directory Interactive` or `Device Code Flow` | `Integrated Security` (`Trusted_Connection`), any `Authentication` other than `SqlPassword`, `Active Directory Password` or `Active Directory Service Principal`, `Column Encryption Setting=Enabled`, a named-pipe (`np:`), shared-memory (`lpc:`) or LocalDB `Data Source` | | MySQL | `AllowLoadLocalInfile=true`, `AllowLoadLocalInfileInPath`, `CommandInterceptors`, `ExceptionInterceptors` | `IntegratedSecurity`, a Kerberos, Windows or OCI `DefaultAuthenticationPlugin`, `OciConfigFile`, `CertificateStoreLocation`, a `ConnectionProtocol` other than TCP, no `Password` | | Snowflake | `client_config_file`, `disable_saml_url_check=true`, `authenticator=externalbrowser` or `oauth_authorization_code` | `authenticator=workload_identity`, `private_key_file` (give the key with `private_key`) | | BigQuery | **Service account JSON** that is not a service account key (`"type": "service_account"`), or a key whose `token_uri` is not `https://oauth2.googleapis.com/token`, whose `universe_domain` is not `googleapis.com`, or whose other `*_uri`/`*_url` is not an https URL on `accounts.google.com` or below `googleapis.com` | No service account JSON (the host's application default credentials) | | AWS CloudWatch | An **AWS region** that is not a region name | An **AWS profile**, or no access key (the host's IAM role) | | Databricks | A **Host** that is not a bare host name (no `https://`, port or path) | | The rule applies where the transport rule does: creating, changing and testing a connection refuse an option above (the message names the option, never a value); a stored source keeps working, with one Warning per data source per process, and is refused only under `Beacon:DataSources:Transport:EnforceOnExisting`. A host-managed source is the host's own wiring and may sign in as the host: the option rule does not apply to it, only the host list below. BigQuery also builds its client from a service account key that reaches only Google, so a stored source with any other credential JSON fails when it connects. `Beacon:DataSources:Transport:AllowInsecure` lifts the option rule on development hosts. A server can also ask for a sign-in the connection does not name. A PostgreSQL connection that carries its own `Password` or `SSL Certificate` opens with `Require Auth=!GSS,!SSPI` and `GSS Encryption Mode=Disable` unless it sets them itself, so a server asking for Kerberos or SSPI is refused before the Beacon host's identity is offered. MySql.Data follows a server's switch to another authentication plugin and has no setting to refuse it: with a password its Kerberos plugin uses the login's own credentials and its OCI plugin needs the OCI SDK (which Beacon does not reference), but a Beacon host on Windows answers a switch to the Windows plugin with its Windows account, so connect from Windows only to MySQL servers you trust. `Beacon:DataSources:AllowedHosts` restricts the hosts a connection may reach: every host the connection names (each host of a PostgreSQL or MySQL host list, the SQL Server failover partner, the Snowflake host or the account's `snowflakecomputing.com` host as the driver composes it, its proxy, Okta and OAuth URLs, the API base and spec URLs, the Databricks host, the BigQuery key's `token_uri` host `oauth2.googleapis.com`) must be listed. CloudWatch reaches only its region's AWS endpoint and passes. A host that is not a bare host name or IP address (a scheme, port, path, `?`, `#`, `@`, a space or a non-ASCII character in it) matches no entry, and a connection whose hosts its connector cannot tell (MySQL `DnsSrv`, a Snowflake OAuth URL that is not an absolute URL, a type with no option check) is refused while the list is set. Entries and hosts are ASCII (write an internationalised name in its `xn--` form) and IP addresses are compared in their canonical form. The list matches names as the connection writes them: the addresses they resolve to are not pinned, so it trusts the Beacon host's DNS. `AllowInsecure` does not lift the list. See [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/#data-source-driver-options-and-hosts). ### Engine check The **Database engine** you pick decides the driver, the SQL grammar and the read-only checks Beacon applies, so Beacon asks the server which engine it is when you **create** a database data source, **change its connection or engine**, and click **Test connection**. It runs one fixed catalog statement, outside the SQL gate, like the metadata scan's catalog SQL: | Declared engine | Statement | Refused when the server is | |---|---|---| | PostgreSQL | `SELECT version()` | not PostgreSQL: the answer does not start with `PostgreSQL`, or names Amazon Redshift, Greenplum, CockroachDB or YugabyteDB | | SQL Server | `SELECT CAST(SERVERPROPERTY('EngineEdition') AS int)` | Azure Synapse (edition 6 or 11) | | Azure Synapse | the same | SQL Server (any other edition) | | MySQL | `SELECT VERSION()` | never: MariaDB and TiDB pass with a note in the test result and a Warning in the log | Snowflake, BigQuery, Databricks, CloudWatch and API sources are not checked. - **A clear mismatch** refuses the create or the change (`400`) and fails **Test connection**, with a message naming the declared and the detected engine, never a version string or a host name. Nothing is saved. - **A server that cannot be reached, or whose answer cannot be read**, never blocks the save: the log carries a Warning naming the data source by id. **Test connection** reports reachability as before; when the server answered the connection but not the engine question, it says *Connection successful. The server's engine could not be verified.* - **An edit that keeps the stored connection and engine** connects to nothing. - **A disabled data source** is not checked (it makes no connection, also when its credentials are rotated): the check runs the next time its connection or engine changes while it is enabled. - A refused connection's pooled connections are emptied, so none opened with the refused credentials is reused. - **Under the approval workflow** (`CoversDataSourceConnections`), a source created or changed in anything but its secrets connects to nothing until approved, so neither the create nor the change runs the check: it runs when an Approver approves the connection. Then a clear mismatch, and a server that cannot be reached or whose engine cannot be read, refuse the approval (`409`) and the request stays pending; a disabled source is enabled first (`409`). A change of the secrets alone is checked when it is saved, as without the workflow. See [Approvals](https://moberghr.github.io/beacon/security/approvals/). ## Manage data sources The **Data Sources** list is paged on the server (20 rows per page, at most 200) and can be searched by name. Its columns are **Id**, **Name**, **Type**, **Engine** (with the read-only badge), **Projects**, **Queries** and **Migrations**. A user who holds DataSourceAdmin but no Viewer or Editor role can add data sources but cannot list them. - **Projects** shows the projects that link the data source and that you may use, plus "+N other" for projects you cannot see (never named). "Admins only" marks a data source no project links. - **Queries** counts only the queries you may read; the migration job count is shown to Admins only. Click a row to open the data source page: type, table, column and query counts, the read-only and migration-write status, **Metadata loading**, the linked projects, and the schema browser. Its actions are **Relationships**, **Refresh metadata** and **Delete**. ### Edit a data source and replace its connection `PUT /beacon/api/data-sources/{id}` (DataSourceAdmin or Admin) changes a data source's name, engine, metadata loading options and connection. It is how credentials are rotated: send the new connection string, and the next connection uses it. The stored connection is never shown, so it is never sent back: a body whose `connectionString` is `null` or empty keeps the stored connection and changes only the other fields. A stored connection Beacon cannot decrypt (encrypted under a key no longer configured) cannot be kept; enter it again. Beacon logs that it could not decrypt it, with the data source id and engine only. `name` is required. A database data source needs `databaseEngineType`, and changing the engine needs the connection entered again: a stored connection is never read as another engine's (`400`). The metadata options are optional: one that is left out or `null` keeps the stored value, and an empty schema list clears the stored one. ```json { "name": "warehouse", "databaseEngineType": "PostgreSQL", "connectionString": "Host=db.example.com;Database=sales;Username=beacon_reader;Password=...;SSL Mode=VerifyFull", "metadataLoadingEnabled": true, "metadataMaxTables": 0, "metadataMaxColumnsPerTable": 0, "metadataLoadTableNamesOnly": false, "metadataExcludeSchemas": [], "metadataIncludeSchemas": [], "confirmTargetChange": false } ``` - **Transport.** A new connection, or the stored one read as another engine's, must authenticate the server over TLS (see [Transport security](#transport-security-tls)); otherwise the request answers `400` naming the option to change. - **Engine.** A new connection or engine is checked against the server (see [Engine check](#engine-check)): a server of another engine than the declared one answers `400` and nothing changes. - **Change of target.** A connection to another engine, host, port or database than the stored one, as the engine's transport check reads them, is saved only when the body sets `confirmTargetChange: true`; otherwise the request answers `409` and nothing changes. A connection whose host, port or database Beacon cannot read (the JSON-configured connectors without a host) counts as a change of target. The host name is compared without regard to case, the database name exactly. A confirmed change removes the column [sample values](#sample-values) stored for the source, in the same save, because they were read from the old target. - **Schema after a change of target.** The stored tables and columns are not deleted: a confirmed change of target enqueues a background metadata refresh (when the source loads metadata) that replaces them with the new target's, keeping the curated descriptions of the tables and columns that still exist. Until it finishes, the schema browser and AI grounding show the old target's tables. A host whose job runner has no metadata refresh job logs that none was enqueued; run **Refresh metadata** then. - **Old connections.** After a connection change, Beacon empties the driver's connection pools for the old connection string on PostgreSQL, MySQL (the read-only pool, which connects without statement batches, included), SQL Server and Azure Synapse, so no idle connection opened with the old credentials is reused. A connection already in use finishes its work and is closed when it is returned. Snowflake and the JSON-configured connectors (Databricks, BigQuery, CloudWatch, API) keep no pool Beacon can empty; for a database engine without one, Beacon logs that no pool was cleared. A pool that cannot be cleared does not undo the saved change; the failure is logged by its type. - **Caches.** Every edit evicts the cached metadata and relationship graph and restarts the forced-refresh cooldown. - **Audit.** A change of connection is recorded as `data_sources.credentials_rotated`, any other edit as `data_sources.data_source_updated`; each names the fields that changed and whether the target changed, never a value (see [Security audit trail](https://moberghr.github.io/beacon/security/audit-trail/)). A data source the host registered (a [host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/) source) cannot be edited here (`400`): the host rotates its credentials in its own configuration. A rotation takes effect at once; it needs no second person's approval. Under the approval workflow (`CoversDataSourceConnections`) that holds for a change of the secrets alone (the password, a key); a change of any other setting of the connection (host, port, database, login, options) is saved and the source connects to nothing until a second person approves it. - **Who may point a source elsewhere.** A DataSourceAdmin may point a data source at another server or database, as one may create a data source there. The change must be confirmed and is recorded with `targetChanged: true`, so the audit trail shows every such change; a second person's approval of a target change is not required unless the approval workflow covers data-source connections. - **Concurrent edits.** Two edits of the same data source saved at the same time are not merged: the one saved last is kept. Queries already running when the connection changes finish on the connection they opened. ### Disable and enable a data source `PUT /beacon/api/data-sources/{id}/enabled` with `{ "enabled": false }` (DataSourceAdmin or Admin) disables a data source at once; `{ "enabled": true }` enables it again. Disabling is the emergency stop for a source whose credentials may be compromised: it takes effect immediately and needs no second person's approval. A disabled source makes no connection at all. The connection resolver, the one place a source's stored connection data is decrypted, refuses it first, before anything is decrypted or its transport checked. So, while it is disabled: - queries, previews, ad-hoc queries, data-quality checks, AI and MCP paths and metadata scans that read it fail with the message "This data source is disabled" (`400`); a data-quality rule result says the same; - a subscription run whose query reads it fails before any step runs, and is recorded in the run history as failed with the reason "Failed: a data source this query reads is disabled." (no task, no delivery); a run someone starts answers `409`; - the project listings that MCP tools and AI grounding read leave it out; - its stored catalog is not served: a metadata read answers an empty catalog (as for a migration write target), and its cached metadata and relationship graph are evicted when it is disabled or enabled, so none of its stored tables, columns or sample values is shown while it is disabled; - the driver's connection pool for its connection is emptied when it is disabled. The data source keeps its queries, subscriptions, projects and metadata, and all of them work again once it is enabled. The list and detail responses carry `disabled` and `disabledTime`. Who disabled it and when are stored on the source; each change is recorded as `data_sources.data_source_disabled` or `data_sources.data_source_enabled`, with the data source id only. Setting the state a source already has changes and records nothing. A host-registered data source cannot be disabled here (`400`). ### Delete a data source **Delete** (`DELETE /beacon/api/data-sources/{id}`) archives the data source (soft delete). It is refused while a live query or migration job still uses it, the message names them, and a data source the host registered (a [host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/) source) cannot be deleted. Archiving evicts the cached metadata and relationship graph; reading or refreshing an archived data source's metadata answers `404`. It also clears the column [sample values](#sample-values) stored for it (archived tables included), so no copy of its rows outlives it. ## Metadata loading options For database data sources Beacon loads tables, columns and relationships. They drive the schema browser, the SQL editor's explorer, the save-time column check, documentation and AI grounding. The options are set when the data source is created: | Option | Default | What it does | |---|---|---| | **Enable metadata loading** | On | Load metadata at all | | **Load table names only** | Off | Skip column details | | **Max tables** | `0` (unlimited) | Cap the number of tables loaded | | **Max columns / table** | `0` (unlimited) | Cap the columns loaded per table | | **Exclude schemas** | none | Comma-separated schemas to skip, for example `information_schema, pg_catalog` | | **Include schemas** | all | Comma-separated schemas to load, for example `public, app` | Turn loading off, or narrow it with schema filters, for very large databases (thousands of tables) and for logins that cannot read the catalog. ### Refreshing metadata **Refresh metadata** on a data source re-reads its tables and columns from the database. It changes what every project that links the data source sees, so it needs Write in each of them, or an Admin; a DataSourceAdmin needs Manage only where it acts (see [Projects and access](#projects-and-access)). A [migration write target](#migration-write-targets) has no refresh, because Beacon never reads it. There are two ways to start one: - **As a background task** (what the page's button does): `POST /beacon/api/data-sources/{id}/metadata-refreshes` answers `202` at once with `{ taskId, jobId, state }` and a `Location` header. Read its progress with `GET /beacon/api/data-sources/{id}/metadata-refreshes/{taskId}` (`state`: `0` queued, `1` running, `2` succeeded, `3` failed, `4` skipped; `failureCode` says why a failed or skipped one did not finish, `warningCount` is the number of names left out), with the same access check. The job runs as the user who started it and checks again, right before the scan, that they may still manage the data source: if not, it is skipped (`forbidden`). A refresh that finds another refresh of the source running in the same Beacon process is skipped too (`busy`; that refresh's result is the current catalog). A signed-in user who started it also gets `JobStatusChanged` events for `jobId` on the SignalR hub (an API key does not). One refresh of a data source is queued or running at a time; one queued or running for more than an hour, or still queued without a job after five minutes, is given up (`abandoned`) when the next one starts or its progress is read. The job runs through your host's [`IBeaconScheduler.EnqueueBackgroundTask`](https://moberghr.github.io/beacon/integrate/background-jobs/); a host that does not implement it runs the refresh in the request, and the `202` then carries the state it finished in. - **In the request** (kept for API callers of Beacon 5.0): `POST /beacon/api/data-sources/{id}/refresh-metadata` answers with the refreshed catalog once the scan is done, which holds the request open for as long as the scan takes. - **Cooldown.** A data source's metadata is force-refreshed at most once per `Beacon:Metadata:ForcedRefreshCooldownMinutes` (default `15`; `0` turns the cooldown off; at most `1440`), whoever asks (Admins and DataSourceAdmins are not held to it) and from whichever project: the cooldown belongs to the data source. A second refresh inside the cooldown, or while one is running, answers `429`. The cooldown counts from the last refresh that succeeded, so a failed refresh can be retried at once. It is kept in memory per Beacon process (for a background refresh, the process whose job ran it): on a host running several instances, each instance keeps its own. A background refresh queued or running anywhere refuses another start on every instance. - **Curated descriptions are kept.** A table or column description set by applying a documentation patch stays; every other description is taken from the database catalog again (and cleared when the catalog has none). - **Names that cannot be quoted safely are left out.** A table or column whose name carries a quote, backtick, bracket, semicolon or control character is not stored, because Beacon builds SQL from catalog names. The refresh answers `skippedNameCount` (a background refresh: `warningCount`; the page says how many were left out), and the server log lists their names: control characters written as `\uXXXX`, at most 50 names. A table left out loses its stored description, a curated one included. - **Sampling is logged per table.** With sample-value collection on, each table that could not be sampled is logged by name with the reason (refused by the execution gate, failed on the data source, timed out, or a name that is not a plain identifier, never the database's own error text), and every refresh ends with how many tables were sampled. - **Caches.** The refresh replaces the cached metadata, and the relationship graph is rebuilt on its next read. ```json { "Beacon": { "Metadata": { "ForcedRefreshCooldownMinutes": 15 } } } ``` ### Sample values A metadata refresh can store a few real values per column (up to five, each cut to 50 characters, or the full set of a short column's values when it has at most 12) to ground text-to-SQL: they are shown to the model as examples. They are row data, so: - **Off by default, and each project's choice.** Sample-value collection (`EnableSampleValueCollection`) is off for a new installation. A refresh collects while it is on in the effective settings of every live project that links the data source (the project's own override, else the global value) so turning it on for a project is enough even while the global value is off, and one linking project that has it off keeps the source unsampled. The global value decides alone only for a data source no live project links. The deployment lock `Beacon:Mcp:ForbidRowDataToLlm` turns it off everywhere (see [MCP Server → What reaches the LLM provider](https://moberghr.github.io/beacon/mcp/server/#what-reaches-the-llm-provider)). - **Columns never read.** The refresh reads only the columns whose values may describe them; its sampling statement names those columns, and a table with none is not queried. A column is left out when its name is PII (the built-in pattern, including the banking identifiers `iban`, `oib` and `jmbg` as a whole name segment: `customer_iban`, `ClientOib`, `IBANNumber`, but not `Libanon` or `Kibana`, and account numbers such as `account_number`, or a custom PII pattern); names a person's name (`first_name`, `LastName`, `surname`, `customer_name`, Croatian `ime`, `prezime`) or Croatian personal data (`adresa`, `telefon`, `mobitel`, `placa`, `stanje`, `broj_racuna`); looks like it holds a secret (`password`, `secret`, `token`, `api_key`, `private_key`, `credential`, `cookie`, `passcode`, `passphrase`, a connection string, `dsn`, `webhook_url`, and the name segments `pwd`, `session`, `signature`, `pin`, `otp`, `salt`, `hash`, `bearer`, `jwt` and `hmac`, so `session_id` and `request_hmac` are left out while `pinned` or `hashtag` are not); has a `key` segment beside `api`, `access`, `secret`, `private`, `signing`, `encryption`, `master`, `license`, `hmac`, `aes`, `rsa`, `pgp`, `gpg` or `ssh`, or is named just `key` (`hmac_key`, `EncryptionKey` and `ssh_key` are left out, while a surrogate key such as `ProductKey` or `ClientKey` is sampled); names a personal financial amount (`salary`, `wage`, `balance`, `income`; not a `balance_type` or an `income_category`); or holds binary data. Financial amounts stay unmasked in query results. - **Values that drop a column's samples.** Every value read is screened whole, before it is cut to 50 characters. When one looks like PII (an e-mail address, a card or phone number, an IBAN, OIB or JMBG with valid check digits, or a custom pattern) or like a credential (a PEM block, a JSON Web Token, a key with a known prefix such as `sk_`, `AKIA`, `ghp_`, `xox…-`, `ATATT`, `AIza` or `glpat-`, a bcrypt, argon2 or crypt hash, a run of more than 32 hex digits, a long mixed-case base64-like run that is not a slug, SKU or path, a credential written into text such as `password=…`, or a URL with a user name and password), the column keeps none. Each refresh logs, as counts only, how many columns it left out by name, for a PII-shaped value and for a credential-shaped value. - **A refresh clears what it no longer samples.** The refresh replaces the stored catalog, so a column that is now left out loses the samples it held. Until then, samples stored earlier are screened again whenever they are read (with today's name and value rules and the custom patterns of every project that links the source) and withheld if they fail. - **Who sees them.** The REST metadata response (`GET /beacon/api/data-sources/{id}/metadata`, and the refresh's response) carries sample values to an Admin only, screened under the settings that apply now; every other caller gets the columns without them (the UI does not show them). Grounding reads the stored samples itself, under the settings of the asking project and the projects it shares the source with: `ask` sends them to the LLM provider, and `get_query_context` returns them to an MCP caller with Read in the project; in both cases only while sample-value collection is on for that project and every project sharing the source, and never under `Beacon:Mcp:ForbidRowDataToLlm`. Both are recorded in the row-data egress log (event `7123`). - **Archiving or a migration write target clears them.** Archiving a data source (see [Delete a data source](#delete-a-data-source)) or marking it as a migration write target clears the samples stored for it, the columns of its archived tables included. ## Schema relationships Loaded metadata tells Beacon what tables exist; **schema relationships** tell it how to join them. Open a data source and click **Relationships** (`/beacon/data-sources/{id}/relationships`) to review and curate them. - **Foreign key** relationships come from the database's declared constraints and are verified by default: one per column pair of every constraint, so a column in several foreign keys (on its own and inside a composite key) gets one for each. - **Inferred** relationships are proposed from column naming with a confidence score. Click **Discover** to preview proposals without saving, then accept the ones that are correct. - **Manual** relationships are declared by hand. One is verified only when its creator may [curate](https://moberghr.github.io/beacon/mcp/knowledge-base/#who-curates-grounding) the data source's shared grounding; otherwise it waits, unverified, for a curator, and a verified relationship that someone else relabels or gives another cardinality goes back to unverified. Only verified relationships are presented to the SQL generator as authoritative join paths; unverified ones are passed separately and flagged. The page opens with a **schema health** panel (table and relationship counts, how many are unverified, connected groups, isolated tables and detected junction tables) which shows which parts of a schema an AI assistant cannot reach. See the [Knowledge Base guide](https://moberghr.github.io/beacon/mcp/knowledge-base/#schema-relationships). ## Projects and access A data source is reached through the projects that link it ([Project access](https://moberghr.github.io/beacon/security/authorization/#data-sources-follow-their-projects)): a caller uses it only through a project they may use, and a query that reads several data sources only through one project that links all of them. Lists are filtered in the database by the same rule. - **Linking.** Link data sources on create (the **Projects** field, `projectIds` on `POST /beacon/api/data-sources`) or from a project's page (**Link data sources**, `POST /beacon/api/projects/{id}/data-sources`), which needs that project's admin role or Admin. Anyone but an Admin or a DataSourceAdmin may link only data sources already linked to a live open project; a DataSourceAdmin who is the project's admin links any existing data source, and gains no membership of the projects that already link it. A data source registered by the host for a host project (a [host DbContext](https://moberghr.github.io/beacon/mcp/host-dbcontext/) source) links only into the project the host declares for it, for Admins too. An unreadable or missing data source answers `404`, a readable one that may not be linked `403`. Each link records who created it. - **Unlinking.** An Admin unlinks a data source from a project (`DELETE /beacon/api/projects/{id}/data-sources/{dataSourceId}`), or from the **Access** card of a project that shares it. When that was the source's last link, the source becomes Admin-only until it is linked again; it never becomes unlinked and open. A host's data source cannot be unlinked (`409`), a missing link answers `404`, and a concurrent change `409` (retry). - **Unlinked data sources.** Data sources that existed before project access stay reachable while no project links them (`openWhenUnlinked`). A new data source is created with `openWhenUnlinked: false`: Admin-only until a project links it. With `Beacon:Authorization:RequireProjectMembership` every unlinked data source is Admin-only. - **Shared data sources.** A data source linked to an open project is reachable through it by everyone with a role, even when a restricted project links it too. The project page warns Admins about such sources before they restrict a project, and lists every other project that links them. - **Shared state.** Relationships, metadata description edits and a forced metadata refresh change what every linking project sees, so they need Write in every live project that links the data source, or an Admin; a DataSourceAdmin needs Write (or Manage) only in a project it acts through. Curating that state (verifying a relationship, deleting a verified one, applying or rejecting a documentation patch that writes a description) needs the project admin role in every live project that links the data source, the DataSourceAdmin permission, or an Admin (see [Who curates grounding](https://moberghr.github.io/beacon/mcp/knowledge-base/#who-curates-grounding)). - **Shared guardrails.** A data source several live projects link runs MCP SQL, and is grounded and sampled, with the strictest guardrails among those projects (read-only enforcement, PII detection and patterns, row limits, value grounding and its probe budget, sample-value collection); an Admin sees them per source on the project page. See [Guardrails of shared data sources](https://moberghr.github.io/beacon/security/authorization/#guardrails-of-shared-data-sources). - **One database login per boundary.** Project access decides who may run SQL through Beacon, not what the login can read. Use a login per data source that sees only what its projects should see, rather than one broad login shared by several boundaries: Beacon refuses names in another database, but a view, synonym or function in the source's own database still reads whatever the login may. ## Read-only guarantees per engine Every statement Beacon runs on a data source first passes its SQL checks (a regex guardrail, a read-only parser check in the engine's dialect, the configured-database rule and a row cap; see [How a query runs](https://moberghr.github.io/beacon/features/queries/#how-a-query-runs)) and then runs on the provider's read-only path. What that path adds at the database differs by engine, and on every engine it is defence in depth, not a replacement for the data source's own permissions: **connect every data source Beacon reads with a login that can only read.** | Engine | What the database itself does on Beacon's read-only path | Data-source badge | |---|---|---| | PostgreSQL | Session `default_transaction_read_only = on` plus a `READ ONLY` transaction, and the statement is sent as a batch of exactly one command, so the driver and the server refuse a second statement: the server refuses any write. | Enforced by the engine; still use a read-only login | | MySQL / MariaDB | `SET SESSION TRANSACTION READ ONLY` plus `START TRANSACTION READ ONLY` on a connection without statement batches, rolled back afterwards: the server refuses any write. | Enforced by the engine; still use a read-only login | | SQL Server | No read-only session or transaction mode exists. The statement runs inside a transaction that is always rolled back, so a write the SQL checks missed is undone rather than committed, but not refused, and work outside the transaction (a sequence value, a call to another server) is not undone. A statement that ends that transaction itself fails the run: its result is discarded and the security log records `ReadOnlyTransactionEnded` (7116, the data source id). | **Read-only login required** | | Azure Synapse | No read-only mode; dedicated and serverless pools differ in what transactions they allow, so the statement runs plainly. | **Read-only login required** | | Snowflake | No read-only session (DDL commits on its own); the statement runs plainly. | **Read-only login required** | | BigQuery | No read-only session; the connector also requires a dry run to report statement type `SELECT`. | **Read-only login required** | | Databricks | No read-only session; the connector re-checks the SQL with its own parser and the configured catalog. | **Read-only login required** | On every engine without a database-level guarantee, Beacon's SQL checks are the only thing between a statement and the data source's own permissions, so a **login (role, service account, token) that can only read** is the boundary. The list and the data source page mark such sources, and Beacon logs a warning (naming the data source by id only) when one is saved and when the host starts, except a [migration write target](#migration-write-targets), which Beacon never reads and labels as one instead. The `databaseReadOnlyEnforced` field of `GET /beacon/api/data-sources` and `GET /beacon/api/data-sources/{id}` reports it: `true` (the engine refuses writes), `false` (it does not; use a read-only login) or `null` (the source runs no SQL, such as CloudWatch or an API source). A SQL Server query over memory-optimized tables runs inside that rolled-back transaction, which such tables accept only under snapshot isolation: if it fails with error 41368, turn on `MEMORY_OPTIMIZED_ELEVATE_TO_SNAPSHOT` for the database. The column sample values collected during a metadata refresh are read on the same read-only path. ## Migration write targets The one exception to read-only logins is a **migration write target**: a data source an Admin marked as the destination of [data migrations](https://moberghr.github.io/beacon/features/data-migration/#migration-write-targets) (data source page → **Migration writes** → **Mark as migration write target**, or `PUT /beacon/api/data-sources/{id}/migration-writes` with `{ "allowMigrationWrites": true }`; Admin only, and marking needs `Beacon:Migrations:Enabled`). **Return to a read source** (or `{ "allowMigrationWrites": false }`) unmarks it. Beacon never reads a write target: query steps, previews, subscriptions, data quality, MCP tools, AI grounding, documentation and metadata scans refuse it or leave it out, and its catalog is not offered, so its login may hold the grants a migration needs to write ([destination privileges](https://moberghr.github.io/beacon/features/data-migration/#destination-privileges)). The list and the page label it **Migration write target** instead of asking for a read-only login, and `allowMigrationWrites` in `GET /beacon/api/data-sources` and `GET /beacon/api/data-sources/{id}` reports it; `?migrationTargets=true` lists only write targets, `false` only the sources Beacon reads. - Only a PostgreSQL, MySQL or SQL Server database can be marked, and not a host-managed data source or one marked read-only. - A data source that live queries, subscriptions or data contracts read is marked only when the Admin confirms it (`"force": true`): otherwise the request is refused with `409` naming how many of each read it, as they fail once Beacon stops reading the source. The page shows that reason and asks a second time. - Marking and unmarking are recorded in the settings history (`DataSource.{id}.AllowMigrationWrites`, before and after, and who) and in the security log (`MigrationWritesChanged`, ids only). - Marking a data source clears the column sample values Beacon collected through its login. - Before you return a write target to a read source, change its login back to one that can only read: from then on Beacon reads it again. The page asks for confirmation and says so. ## Read-only logins Create a dedicated read-only login for every data source Beacon reads (every one except a [migration write target](#migration-write-targets)). On SQL Server, Azure Synapse, Snowflake, BigQuery and Databricks this is the only database-side guarantee that Beacon cannot write. **PostgreSQL:** ```sql CREATE USER beacon_readonly WITH PASSWORD 'strong-password'; GRANT CONNECT ON DATABASE your_database TO beacon_readonly; GRANT USAGE ON SCHEMA public TO beacon_readonly; GRANT SELECT ON ALL TABLES IN SCHEMA public TO beacon_readonly; ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO beacon_readonly; ``` **SQL Server:** ```sql CREATE LOGIN beacon_readonly WITH PASSWORD = 'strong-password'; USE your_database; CREATE USER beacon_readonly FOR LOGIN beacon_readonly; ALTER ROLE db_datareader ADD MEMBER beacon_readonly; ``` Beacon refuses linked-server (four-part) names, and a three-part `database.schema.table` name unless it names the data source's own configured database. A synonym or view can still reach other databases the login can see, so don't map the Beacon login on any linked server, don't grant it `ALTER ANY LINKED SERVER`, and give it access only to the databases Beacon should read. **MySQL:** ```sql CREATE USER 'beacon_readonly'@'%' IDENTIFIED BY 'strong-password'; GRANT SELECT ON your_database.* TO 'beacon_readonly'@'%'; ``` **Snowflake:** a role with `USAGE` on the warehouse, database and schemas and `SELECT` on the tables (no `OWNERSHIP`, `INSERT`, `UPDATE`, `CREATE` or `OPERATE`), set as the user's default role or as `role=` in the connection. **BigQuery:** a service account with `roles/bigquery.dataViewer` on the datasets and `roles/bigquery.jobUser` on the project, not `dataEditor` or `admin`. **Databricks:** a token for a principal with `USE CATALOG`, `USE SCHEMA` and `SELECT` only, and `CAN USE` on the SQL warehouse. **CloudWatch:** Beacon's own CloudWatch features read only the log groups the data source configures, so restrict the access key's IAM user or role to `logs:StartQuery`, `logs:GetQueryResults` and `logs:StopQuery` on those log groups, plus `logs:DescribeLogGroups` for **Test connection**. CloudWatch sources cannot be queried through MCP `query`. ## Troubleshooting **Test connection fails.** Check network reachability from the Beacon host (`nc -vz db-host 5432`, `1433`, `3306`), the server's remote-connection and TLS settings, and the credentials. A message that starts with "Connection refused" names the transport option to change (see [Transport security (TLS)](#transport-security-tls)) or the engine the server actually runs (see [Engine check](#engine-check)). **Certificate errors.** The server's certificate must chain to a CA the Beacon host trusts and name the host in the connection string. Install the issuing CA on the Beacon host, or pin it in the connection (`Root Certificate` on PostgreSQL, `SslCa` on MySQL, `HostNameInCertificate` or `ServerCertificate` on SQL Server and Azure Synapse). Do not turn certificate checks off: Beacon refuses such a connection. **Permission denied on a table.** Check the login's grants: ```sql -- PostgreSQL SELECT grantee, table_schema, table_name, privilege_type FROM information_schema.role_table_grants WHERE grantee = 'beacon_readonly'; ``` **Delete is refused.** Live queries or migration jobs still use the data source; the message names them. **Refresh answers 429.** The data source was refreshed within the cooldown; try again later. ## Related - [Queries](https://moberghr.github.io/beacon/features/queries/): run SQL against a data source - [Data Migration](https://moberghr.github.io/beacon/features/data-migration/): copy data into a migration write target - [Knowledge Base & Grounding](https://moberghr.github.io/beacon/mcp/knowledge-base/): what Beacon builds from this metadata - [Data-source connectors](https://moberghr.github.io/beacon/integrate/connectors/): registering connectors in the host --- # Queries > Saved, versioned SQL: one or more steps across data sources, bound parameters, an optional in-memory final query, and an optional four-eyes review of every change. Source: https://moberghr.github.io/beacon/features/queries/ ![Query editor](https://moberghr.github.io/beacon/img/screenshots/query-editor-light.png) A query is saved SQL that Beacon runs for you. It has one or more **steps**, each a single read-only `SELECT` against one data source, and optionally a **final query** that joins the steps' results in memory. Subscriptions schedule queries, saved-query MCP tools expose them, and AI actors write them. Typical uses: - **Data validation**: return the rows that break a rule (NULLs in required columns, orphaned rows, duplicates, impossible states); a subscription alerts when rows come back. - **Operational health**: row counts, backlog sizes, last-load times, database statistics. - **Scheduled reporting**: a result delivered to people on a schedule. ## Create a query 1. Open **Queries** (`/beacon/queries`) and click **New query**. 2. Under **Query details**, enter a **Query name** and a **Description** (both required; the name is shown in lists and notifications). 3. In the **Cross-data-source query builder**, fill in the first step: a **Step name**, the **Target database** (the data source the step runs on) and the **SQL query**. Click **Add step** for more steps. 4. Click **Save query**, or **Save & run** to save and run it at once. Creating a query needs no project; saving its steps needs Write on the query and on every data source the steps read. The query is then reached through the projects that hold all of its data sources (see [Project access](https://moberghr.github.io/beacon/security/authorization/#project-access)). With the [approval workflow](#approval-workflow) on, the steps you save are submitted for review and the new query runs nothing until a second person approves them. ```sql -- Orders marked completed but not paid: any row is a problem. SELECT id, status, payment_status FROM orders WHERE status = 'completed' AND payment_status <> 'paid' ``` ## Query parameters Write a parameter as its name in single braces: `{start_date}`. The editor detects every `{name}` in a step's SQL and lists it as a parameter of that step (default type String); you can set each parameter's type (Number, DateTime, String), description and placeholder text, or add one by hand. A subscription supplies a value for each parameter, and checks when it is saved that the value reads as the parameter's type. A placeholder is `{`, a name of letters, digits and underscores (it must start with a letter or underscore, so `{1}` is refused although the editor's auto-detect accepts digits; 63 characters at most) and `}`, with no spaces. An empty placeholder is refused, and two parameters of a step cannot share one. Beacon enforces this when a query is created or edited (every step of a multi-step save, and the legacy single-step form), when an AI actor proposes a change, and when a query is newly exposed as an MCP tool; every placeholder must also appear in its step's SQL. Queries stored before the rule keep working unchanged, and restoring an old version keeps its placeholders as stored, also on an exposed tool; editing a query asks for a valid placeholder, and an AI actor cannot refine such a query until a person has edited it. Beacon never pastes the value into the SQL. It replaces the placeholder with a bound database parameter (`@p0`, `@p1`, …) and sends the value separately, as text. So: - **Do not quote the placeholder.** `'{start_date}'` becomes the string literal `'@p0'`, not the value. Write `{start_date}`. - **A value is data, not SQL.** `CURRENT_DATE - INTERVAL '7 days'` as a value is the literal text, never evaluated. Put relative dates in the SQL itself, and keep parameters for fixed values. - **Cast where the engine needs it.** The value arrives as text; cast it when the column's type requires it. ```sql -- PostgreSQL SELECT COUNT(*) AS large_orders FROM orders WHERE amount > CAST({threshold} AS numeric) AND created_at >= CAST({start_date} AS date) ``` ## Multi-step queries Each step runs on its own data source, in order; a query stops at the first step that fails. A query without a final query returns its **last step's** result. A **final query** joins the steps' results: Beacon loads each step's rows into an in-memory SQLite database as the table `@result1`, `@result2`, … (by step order), and runs the final query there. This is how a query compares or joins data from different databases; Beacon never joins across connectors at the database. ```sql -- Final query over two steps on different databases SELECT a.customer_id, a.total AS billed, b.total AS paid FROM @result1 a LEFT JOIN @result2 b ON b.customer_id = a.customer_id WHERE b.total IS NULL OR b.total <> a.total ``` The query editor shows the final query read-only; set it through the API (`finalQuery` on `PUT /beacon/api/queries/{id}`) or have an [AI actor](https://moberghr.github.io/beacon/features/ai-actors/) write it. The final query is SQLite SQL and passes the same execution gate as every step (see [How a query runs](#how-a-query-runs)). ## Manage queries ### The Queries list The list shows **Name** (with the description), **Source** (user-defined, or the AI actor that wrote it), **Steps**, **Subscriptions** and **Created**. It is searchable by name and paged on the server (20 rows per page, at most 200), and it holds only the queries you may read. Click a row to open the query. ### The query page The query page (`/beacon/queries/{id}`) shows the steps, the final query and run statistics, with these actions: | Action | What it does | |---|---| | **Execute query** | Runs the saved query now and shows the result (`⌘`+`↵`). Needs the Run action on the query. | | **Add subscription** | Opens the new-subscription dialog for this query. | | **Edit SQL** | Opens the editor (`/beacon/queries/{id}/edit`). | | **Lock** / **Unlock** | Locks the query against AI actor changes and against restores by anyone but the lock holder or a caller who may manage it; it does not block a person's ordinary edits. Locking needs Write; while a query is locked, restoring it needs Manage, except for the lock holder. Only the lock holder or an Admin can unlock it (`403` for anyone else); a lock with no recorded holder is lifted by anyone with Manage. | The tabs below list the query's **Subscriptions**, **Executions**, **Versions** and **Change history**. The **MCP tool** card shows whether the query is exposed as a [saved-query MCP tool](https://moberghr.github.io/beacon/mcp/saved-query-tools/); a SecurityAdmin (or an Admin) turns it on or off there. ### Edit a query The editor changes the **Name**, **Description** and the steps: **Add step**, **Move up**, **Move down**, **Delete step**, and each step's name, data source, SQL and parameters. Saving writes a new active version, or, with the approval workflow on, submits the change for review. The editor sends the version it loaded, so a save made on top of a version that is no longer active is refused (`409`); reload and make the change again. Before a step is stored, its SQL must pass the execution gate, and when Beacon holds metadata for the step's data source, every column the SQL reads from a known table must exist there (refresh the data source's metadata after adding a column). The same check applies when an AI actor writes a query. ### Versions and change history Every executable change is a **version**: Active (live), Archived (replaced), PendingApproval, Rejected or Superseded. **Versions** (`/beacon/queries/{id}/versions`) lists them; a version's page shows its steps and final query and compares it with the current version (`GET /beacon/api/query-versions/diff?versionIdA=…&versionIdB=…`). **Restore** (`POST /beacon/api/query-versions/{id}/restore`) puts back a version that was live (Active or Archived) as a new version, or submits it for approval while the workflow is on. **Change history** lists the query's recorded changes. ### Folders Queries can be filed in folders through the API: `GET`/`POST /beacon/api/query-folders`, `PUT` and `DELETE /beacon/api/query-folders/{id}` (delete optionally moves the folder's queries to its parent), and `PUT /beacon/api/queries/{id}/folder` to move a query. The Queries list does not show folders. ### Deleting Queries cannot be deleted, in the UI or the API. Stop a query from running by archiving its subscriptions, and remove an MCP tool by turning it off on the query page. While a query is live, the data sources it reads cannot be deleted. ### Who created and changed it A query records who **created** it and who **changed it last**, and when, taken from the caller, never from the request: an edit, restoring a version, archiving it, and, with the [approval workflow](#approval-workflow) on, the approval that makes a proposed version live (the reviewer is recorded as the last to change it). Queries created before this was recorded show neither. An Auditor or a UserAdmin finds the queries a user owns, created or changed last under [Owned resources](https://moberghr.github.io/beacon/security/user-management/#owned-resources). The submissions and decisions of the approval workflow are recorded in the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/#event-catalogue) (`approvals.query_change_*`); creating and editing a query outside the workflow is attributed on the query but not recorded there as an event. ## Approval workflow The approval workflow is **off by default**. A host turns it on in code (`options.EnableApprovalWorkflow(...)` in `AddBeaconServices`); see [Approvals](https://moberghr.github.io/beacon/security/approvals/) for the switches, the review rules, the Approvals page and the endpoints. With it on, a change to what a query runs goes live only after a second person approves it. - **What counts as a change.** The query's executable content: its steps (added, removed, reordered), each step's name, data source, SQL and parameters (name, type, placeholder), and the final query. Every character of the SQL counts, whitespace and line endings included. A change to the query's name or description alone goes live directly. - **Submitting.** Saving files a *PendingApproval* version holding exactly the submitted content and its content hash (SHA-256 of the executable content), plus an approval request naming you as the requester. The live query is not touched. A query has one pending request: your new submission supersedes your own earlier one, and while another person's request waits, every submission of the query is refused (`409`) until that one is approved, rejected or withdrawn. - **New queries** have nothing live until their first version is approved. - **Deciding.** An **Approver** (an Admin holds the permission) other than the requester (and other than anyone who shaped the change, such as the reviewer of an AI actor's plan) approves on the **Approvals** page (`/beacon/approvals`). The approval names the content hash the reviewer was shown (`expectedContentHash`) and applies only on top of the version the change was made on; otherwise it answers `409`. The requester may withdraw their own request by rejecting it. Hosts with a single approver can allow self-approval (`AllowSelfApproval`). - **Restores and AI actors** are submitted like a save. An AI actor's SQL is validated without running it, and its creator is the requester (see [AI Actors](https://moberghr.github.io/beacon/features/ai-actors/#query-change-approvals)). **Runs execute reviewed SQL.** With the workflow on, a subscription run (scheduled, a test delivery, or one an AI actor created) and a [saved-query MCP tool](https://moberghr.github.io/beacon/mcp/saved-query-tools/#which-version-runs) execute only the content of the query's active version. When the stored steps are not that content, nothing runs: the run is recorded as **Skipped** with "Skipped: query change not approved. Save the query to submit its SQL for approval, or restore a version and approve it.", the subscription page shows a banner, its Control Tower row reports `notApproved`, and a test delivery answers `409`. Saving the query brings such SQL under review. Interactive runs (**Execute query**, the editor's previews) run the caller's SQL under the caller's own permissions and are not compared. In **strict mode** (`RequireApprovedVersionToRun`, recommended for regulated deployments) a run also needs the active version to carry an approved request for exactly its content, so a version that went live while the workflow was off does not run until it is approved (restore it to submit it). `IQueryService.AddQueryStep` and `UpdateQueryStep` change one live step without a version or a review: they are refused (`409`) while the workflow is on, and are obsolete; save the query instead. ## Database names A step's SQL may name objects of its own data source's database only. Unqualified and schema-qualified names always work; a name qualified with a database must use the one the data source's connection is configured for, and is refused otherwise, when the query or step is saved, and again before every preview, run and subscription execution: | Engine | Full name | Must equal | |---|---|---| | SQL Server, Azure Synapse | `database.schema.table` (four-part linked-server names are refused) | `Initial Catalog` / `Database` | | PostgreSQL | `database.schema.table` | `Database` / `DB` (unquoted names fold to lower case) | | MySQL | `database.table` | `Database` / `Initial Catalog` | | Snowflake | `database.schema.table` | `db` (unquoted names fold to upper case) | | Databricks | `catalog.schema.table` | the configured `Catalog` | | BigQuery | `project.dataset.table` (also inside one backtick-quoted name) | the configured `ProjectId` | Function and type names follow the same rule. A data source without a configured database accepts no database qualifier, and names resolved only at run time (`IDENTIFIER(...)`, Snowflake `TABLE('...')`) are refused. The engine's catalog, statistics and query history (`information_schema`, `pg_catalog` and `pg_…` relations, `sys`, Snowflake `QUERY_HISTORY` and `RESULT_SCAN`, BigQuery `INFORMATION_SCHEMA`, …) are refused too, also inside the source's own database. The final query of a multi-step query runs on the in-memory step results and is not affected. See [Database names in SQL](https://moberghr.github.io/beacon/security/authorization/#database-names-in-sql). ## How a query runs Every statement Beacon runs for a query (a preview (stored, draft or paged), a subscription run or its test delivery, an AI actor's check of the query it wrote (with the approval workflow off; with it on that check runs nothing, see [AI actors](#approval-workflow)), a data migration's source read) goes through the same execution gate as the MCP tools, then the data source's read-only path: 1. **The execution gate**, with read-only enforcement always on: a length cap, a refusal of control characters (other than tab and line breaks) and Unicode noncharacters, the regex guardrail, the read-only parser in the data source's dialect (one `SELECT`, no writes, no locking hints, no side-effecting or cross-connection functions), the [database-name rule](#database-names), the refusal of the engine's catalog, statistics and query history, the host policy for a host data source, and a row cap written into the SQL itself (`LIMIT` / `TOP`). The text judged is the text that runs: placeholders bound as parameters, `--` line comments removed (as the data source's dialect reads them) and line breaks flattened, so a comment line never swallows the clause after it. The regex guardrail reads the statement without its comments: a step may start with `-- Revenue by month` or explain itself with `/* … */`, even naming a write it does not make, while MySQL executable comments (`/*! … */`) and optimizer hints (`/*+ … */`) stay and are judged; the parser always reads the SQL as written. An `END` that closes no `CASE` is refused (the parser would stop reading there), and on SQL Server and Azure Synapse so is an unquoted alias that is a statement keyword, such as `SELECT 1 COMMIT`: quote it (`[COMMIT]`) to use it as a name. 2. **The read-only path** of the provider: on PostgreSQL and MySQL the statement runs inside a `READ ONLY` session and transaction (on PostgreSQL as a batch of exactly one command, so a second statement is refused), so the database itself refuses a write the parser missed. SQL Server has no such mode: the statement runs inside a transaction that is always rolled back, which undoes such a write but does not refuse it, and a statement that ends that transaction fails the run. Azure Synapse and Snowflake run it plainly; BigQuery and Databricks refuse anything their own dialect's parser does not read as a single `SELECT`, and BigQuery runs only what a dry run reports as a `SELECT`. On every engine but PostgreSQL and MySQL a read-only login is the database-level boundary, and the data source pages say so; on PostgreSQL and MySQL the engine's read-only session is defence in depth, and a read-only login is still required ([Read-only guarantees per engine](https://moberghr.github.io/beacon/features/data-sources/#read-only-guarantees-per-engine)). 3. **Limits:** one statement may return at most `Beacon:Query:MaxExecutionRows` rows (default 100,000; see [Query Execution Row Cap](https://moberghr.github.io/beacon/getting-started/configuration/#query-execution-row-cap)) (the cap is written into the SQL: a statement's own `LIMIT`, `TOP` or `FETCH` above the cap, `LIMIT ALL` or `NULL` is lowered to it, and an expression, `TOP … PERCENT` or `WITH TIES` is wrapped in a capped derived table (on SQL Server a statement with a `WITH` clause that cannot be lowered is refused); on SQL Server and Azure Synapse a `UNION`, `EXCEPT` or `INTERSECT` without an `ORDER BY` is capped as a derived table too, so its result columns need names (alias them). On PostgreSQL, MySQL, SQL Server, Azure Synapse and Snowflake the read stops one row past the cap, BigQuery stops reading the job's result there, and Databricks is sent the same bound as its `row_limit`. A larger result fails the run with *returned more than N rows (the `Beacon:Query:MaxExecutionRows` limit)* instead of delivering part of it) and a statement may run for `Beacon:Query:StatementTimeoutSeconds` (default 120 seconds). Closing a preview or cancelling a request stops the statement. The final query of a multi-step query runs on the steps' results in Beacon's in-memory SQLite store, behind the same gate (without the database-name rule, it reaches no database) and the store's own read-only engine settings. A final query that runs out of time fails the run, a preview, a subscription, an AI actor's check and a migration all see a failure, never an empty result. ## Troubleshooting **"Step N returned more than N rows (the `Beacon:Query:MaxExecutionRows` limit)".** A larger result fails instead of being cut short, so a subscription never reports a partial count. Filter or aggregate the step, or raise the setting as far as the host's memory allows (a migration's read names its own limit, `Beacon:Migrations:MaxRows`). A paged preview reads one page and a count when the SQL's shape allows it; SQL it cannot page is read whole and previewed only up to the limit. **Timeout.** A statement may run for `Beacon:Query:StatementTimeoutSeconds` (default 120 seconds). Check the execution plan, add indexes, or narrow the step. **"Column does not exist on a table".** The save check found a column the data source's stored metadata does not list. Check the name, or refresh the data source's metadata if the column was added after the last refresh. **"The SQL names an object in another database".** A name is qualified with a database other than the one the step's data source is configured for; "has no database configured" means the connection names none. Write the name as `schema.table` (MySQL: `table`), or configure the database on the data source. See [Database names](#database-names). **"… is a system catalog relation".** The step reads the engine's catalog, statistics or query history (`information_schema`, `pg_catalog`, `pg_stats`, `sys`, …), which Beacon refuses on every data source. Read the source's own tables instead; its tables and columns are listed from Beacon's metadata. **A parameter does not filter.** Check that the placeholder is not quoted (`{name}`, not `'{name}'`) and cast it when the column is not text. **"Another change to this query is waiting for review".** Someone else's change is pending; it must be approved, rejected or withdrawn first. See [Approval workflow](#approval-workflow). **A subscription run shows Skipped: query change not approved.** The stored SQL is not the approved active version. Save the query to submit it, or restore a version and have it approved. ## Related - [Subscriptions](https://moberghr.github.io/beacon/features/subscriptions/): schedule a query and deliver its results - [Approvals](https://moberghr.github.io/beacon/security/approvals/): the change-review workflow - [Saved query tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/): expose a query to MCP clients - [Data Sources](https://moberghr.github.io/beacon/features/data-sources/): connections and read-only guarantees --- # Subscriptions > Run a query on a cron schedule, decide which runs notify whom, and choose what each notification carries: runs execute as the subscription's owner. Source: https://moberghr.github.io/beacon/features/subscriptions/ A subscription runs one query on a schedule and delivers the result to recipients. It decides when the query runs, which runs notify anyone, what each notification carries, and whether runs open [tasks](https://moberghr.github.io/beacon/features/tasks/) or check for [anomalies](https://moberghr.github.io/beacon/features/anomaly-detection/). A subscription has no name of its own: lists, notifications and Jira issues use its query's name. ## Create a subscription Open **Subscriptions** (`/beacon/subscriptions`) and click **New subscription**, or click **Add subscription** on a query's page. Creating one needs Write on the query (and, for an API key, the Execute scope). The dialog has five steps. ### 1. Query | Field | Default | What it does | |---|---|---| | **Query** | Required | The query to run (required). | | **Cron expression** | `0 9 * * *` | When it runs: a standard 5-field cron expression (required). Runs may be no closer than the deployment's minimum interval (5 minutes by default); see [Cron schedules](#cron-schedules). | | **Query parameters** | None | A value for each of the query's [parameters](https://moberghr.github.io/beacon/features/queries/#query-parameters), used by every run. Each value must read as the parameter's type. | Parameter values are bound as data. A value such as `CURRENT_DATE - INTERVAL '7 days'` is the literal text, never evaluated: write relative dates into the query's SQL instead. ### 2. Notify | Field | Default | What it does | |---|---|---| | **Send notification** | When result count changes | Which runs notify the recipients: **When result count changes**, **Every execution**, or **When result count increases**. | | **Minimum row count** | none | Notify only when a run returns at least this many rows. | | **Recipients** | None | Who is notified. Required unless **Create a task** is ticked. Recipients are created on the [Recipients](https://moberghr.github.io/beacon/features/notifications/#recipients) page. | | **Create a task and keep it open while the query returns rows** | off | The first run with rows opens a [task](https://moberghr.github.io/beacon/features/tasks/); later runs update it, and a run with no rows resolves it. This happens on every run, whatever the notification setting. | ### 3. Results | Field | Default | What it does | |---|---|---| | **Max rows** | default cap (10,000) | At least 1; empty or a stored 0 uses the default. The most rows that reach recipients: the inline table and the attachment. The run's record count is always the full count. | | **Timeout (seconds)** | the deployment's limit | How long each statement of a run may take, the in-memory final query included. Empty uses `Beacon:Query:StatementTimeoutSeconds` (default 120). A longer value than that limit is refused when you save. | | **Include results as attachment** | off | Attach the result to the notification as **CSV** or **XLSX** (**Attachment format**). Only email delivers attachments. A file larger than `Beacon:Subscriptions:MaxAttachmentBytes` (10 MB by default) keeps only its first rows that fit (a CSV exactly as many as fit; an XLSX file, which compresses unevenly, can keep somewhat fewer), and the email says how many of the rows it delivers the attachment holds. When the first row alone is larger than the limit, the file has the column headers only; when even the headers are larger, nothing is attached. The email says which. Email encoding makes the message about 1.33 times the attachment's size. | | **Show query text in notification** | off | Include the SQL in the message. | | **Store result rows for later viewing** | off | Keep a preview of the result's first rows on the run's record, shown on the notification's page. | ### 4. Anomaly **Enable anomaly detection** makes the subscription notify only when a run's row count is unusual against its history, instead of using the **Send notification** rule. Set the **Detection method**, **Sensitivity**, **Lookback (days)** (the dialog accepts 7–365, default 30), **Minimum data points** (the dialog accepts 3–100, default 7), and whether to alert on an unusual increase, decrease or both. See [Anomaly Detection](https://moberghr.github.io/beacon/features/anomaly-detection/). ### 5. Review Check the summary and click **Create subscription**. The subscription records you as its [owner](#owner-and-project-access) and starts running on its schedule. ## Which runs notify Every run is recorded with a status. Recipients are notified only when the status is **Sent**: | Status | When | |---|---| | **Timeout** | A step did not complete: it ran out of time or failed. | | **No results** | The query returned no rows. Nobody is notified, whatever the trigger. | | **Below threshold** | Fewer rows than **Minimum row count**. | | **Sent** | The trigger fired: **Every execution**; **When result count changes** and the count differs from the last run's; **When result count increases** and it is higher. A run with no earlier run to compare with notifies. With anomaly detection on: an anomaly was found. | | **Silenced** | The trigger did not fire, or anomaly detection found nothing unusual. | | **Failed** | Delivery to at least one recipient failed. The others are still notified; the run names the failed recipient ids with a generic reason. | | **Skipped** | Nothing ran, or nobody could be notified; see [Scheduled runs run as the owner](#scheduled-runs-run-as-the-owner). | A skipped run is not a baseline: the result-count trigger compares a run with the last run that was not skipped. ## Cron schedules Five fields: minute, hour, day of month, month, day of week. Beacon validates the expression when you save; the host's job runner (see [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/)) fires the runs. A six-field expression with seconds is refused. Two runs may be no closer than `Beacon:Subscriptions:MinIntervalMinutes` (default 5 minutes): saving a schedule that fires more often, such as `* * * * *` or `0,1 9 * * *`, is refused with a message naming the key. The gap counts across the hour and across midnight too (`58,3 * * * *` runs 5 minutes apart at :58 and :03). The schedule is evaluated in UTC, as the job runner fires it. A subscription saved before the limit existed keeps its schedule: it still runs, an edit that keeps the schedule is accepted, and a reactivation puts it back on the scheduler unchanged. Only a change of the schedule is checked. A subscription an [AI actor](https://moberghr.github.io/beacon/features/ai-actors/) creates is held to the same limit. The check covers the whole 400-year calendar cycle, so days that fall in a row only once in decades (28 and 29 February on a Friday and a Saturday) count too. | Expression | Runs | |---|---| | `*/5 * * * *` | Every 5 minutes | | `0 * * * *` | Every hour, on the hour | | `0 */6 * * *` | Every 6 hours | | `0 9 * * *` | Daily at 09:00 | | `0 9 * * 1-5` | Weekdays at 09:00 | | `0 9,13,17 * * *` | At 09:00, 13:00 and 17:00 | | `*/15 8-18 * * *` | Every 15 minutes from 08:00 to 18:59 | | `0 9 * * 1` | Mondays at 09:00 | | `0 9 1 * *` | The first day of every month at 09:00 | | `0 9 1 1,4,7,10 *` | The first day of every quarter at 09:00 | ## Manage subscriptions ### The Subscriptions list The list shows **Id**, **Query**, **Query name**, **Schedule**, **Recipients** and **Source** (Direct, or the AI actor that created it), filtered to **Active** or **Archived** and searchable by query name. It is paged on the server (20 rows per page, at most 200) and holds only the subscriptions you may read. Click a row to open the subscription. ### The subscription page The page (`/beacon/subscriptions/{id}`) shows the query configuration, the owner, the notification settings and run statistics, with the tabs **Recipients**, **Anomaly detection** (shown only when anomaly detection is on) and **Execution history**. | Action | What it does | Needs | |---|---|---| | **Test now** | Runs the subscription once, now, and delivers to its recipients (`POST /beacon/api/subscriptions/{id}/execute`). It runs as you, not as the owner, and does not change the schedule. | Write and Run | | **Archive** | Stops the schedule and moves the subscription to the Archived list (`DELETE /beacon/api/subscriptions/{id}`). Its runs stay in the history (listed with `subscriptionArchived: true`). | Manage. The button is enabled for Admins; others with Manage use the API. | | **Reactivate** | Puts an archived subscription back on its schedule (`POST /beacon/api/subscriptions/{id}/reactivate`). | Write. The button is enabled for Admins; others with Write use the API. | | **Change owner** | Assigns another owner (see below). | Admin or UserAdmin | | **Add recipient** / **Remove recipient** | On the **Recipients** tab (`POST /beacon/api/subscriptions/{id}/recipients`, `DELETE …/recipients/{recipientId}`). | Write | There is no pause and no edit of a subscription's schedule, timeout, parameters, trigger or result options. To change them, create a new subscription and archive the old one. What can change after creation is its recipients, its owner, and its SLA. ### SLA hours A subscription's SLA sets how long its open tasks may stay unresolved; the task page counts down against it (24 hours when none is set). An Admin edits it from the SLA banner on a task page (the **Edit SLA** gear: 1–720 hours, or **Use default**). Through the API, `POST /beacon/api/subscriptions/{id}/sla` with `{ "slaHours": 8 }` (1–720, or `null` to clear) needs Write on the subscription. ## Owner and project access A subscription records its **owner** when it is created: the signed-in creator, or, for a subscription an [AI actor](https://moberghr.github.io/beacon/features/ai-actors/) creates, the actor's creator, never a value from the request. The page shows the owner by display name, or as "User {id}" without one, never a user name or external id; subscriptions created before owners were recorded show "no owner recorded", and an owner who matches no Beacon user any more shows "unknown user". While Beacon user management is on, a caller without a Beacon user (a host principal, or a bearer token not bound to a user) cannot create a subscription (`403`). A subscription belongs to the projects that hold its query's data sources: it, its runs and their stored results are visible only through such a project you may use ([Project access](https://moberghr.github.io/beacon/security/authorization/#project-access)). One you may not read answers `404`. ### Scheduled runs run as the owner Every scheduled run runs as the subscription's owner, as they are at that moment: right before any SQL runs, Beacon checks that the owner may still run saved SQL in one project that holds every data source of the query's steps. When the owner may not (disabled or archived, removed from a restricted project, or without the role that runs saved SQL) the run is **skipped**: - nothing runs against the data sources, no notification is sent, no task is created or updated, and an AI actor is not triggered; - the run is recorded as **Skipped** with "Skipped: owner not authorized."; - the job does not fail, and a skipped run is not counted as an execution failure (the Home page's activity feed lists it as skipped); - the Control Tower counts it as an unsuccessful run and marks the subscription (`ownerNotAuthorized`), as does the subscription's detail. The subscription page shows a banner until an Admin or UserAdmin assigns an owner who has access. A subscription without an owner (created before owners were recorded) runs only while every data source of its query is openly reachable; otherwise it is skipped, and with `Beacon:Authorization:RequireProjectMembership: true` it is always skipped. Runs are also **Skipped** when, with the [approval workflow](https://moberghr.github.io/beacon/security/approvals/) on, the query's stored SQL is not its approved active version (the page shows "The last run was skipped" with the remedy; see [Queries → Approval workflow](https://moberghr.github.io/beacon/features/queries/#approval-workflow)), and when every recipient is waiting for approval of where it delivers (see [Notifications](https://moberghr.github.io/beacon/features/notifications/#approval-of-new-and-changed-destinations)). ### Change the owner An Admin or a UserAdmin clicks **Change owner** on the subscription page (in the header, and in the "owner not authorized" banner), picks a user and saves, or calls `PUT /beacon/api/subscriptions/{id}/owner` with `{ "userId": 12 }`. The new owner must be an existing, enabled user who may run the subscription's query in one project that holds all of its data sources; anyone else is refused with `409`, because every run would be skipped. The change is made only from the owner it was checked against: when someone else changed the owner (or archived the subscription) in the meantime, nothing is changed and the call answers `409`; reload and try again. Choosing the owner the subscription already has changes nothing, records nothing and checks nothing. ### Who created and changed it A subscription also records who **created** it (the signed-in creator; for one an AI actor creates, the actor's creator) and who **changed it last**, and when: an edit, adding or removing recipients, a new SLA or owner, archiving and reactivating it, and archiving its query. Like the owner, these are taken from the caller, never from the request, and subscriptions created before this was recorded show neither. They are not shown on the subscription page; an Auditor or a UserAdmin finds a user's subscriptions under [Owned resources](https://moberghr.github.io/beacon/security/user-management/#owned-resources). Creating, editing, archiving and reactivating a subscription, adding and removing its recipients, changing its SLA and changing its owner are recorded in the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/#event-catalogue) (`delivery.subscription_*`, `ownership.owner_changed`), by ids, never a recipient's destination. ## What a run reads and delivers A run executes its query through the same execution gate and read-only path as every other query execution (see [How a query runs](https://moberghr.github.io/beacon/features/queries/#how-a-query-runs)): - **Read-only:** every statement passes the regex guardrail, the read-only parser in the data source's dialect and the database-name rule, and runs inside a `READ ONLY` transaction on PostgreSQL and MySQL. - **Rows:** a statement may return at most `Beacon:Query:MaxExecutionRows` rows (default 100,000). A larger result fails the run rather than delivering part of it, so the run's record count (which drives the trigger, the minimum row count, anomaly detection and tasks) is always exact. **Max rows** then limits how many of those rows reach recipients. - **PII masking:** before the rows reach recipients, attachments or the stored result, columns holding PII are masked: by the PII names in the SQL, by their own result-column names and by aliases over PII columns, following the PII detection setting and custom patterns of the projects that link the query's data sources (the strictest applies, as for MCP). With PII detection off in every linking project, rows are delivered as read. With a final query, the steps' rows are joined as read and the final query's result is masked instead: every column it fills from a step's PII column, by name, by an alias or by an expression over it, and every column when its select list cannot be traced. Columns a host data source masks arrive masked and stay masked. A run that fails is recorded with its status, and the next scheduled run still happens. ## Troubleshooting **The subscription never runs.** Check that it is Active (not Archived), that the cron expression is what you meant, and that the host's job runner is running (see [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/)). **Runs show Skipped.** Read the run's reason: the owner is not authorized (change the owner), the query change is not approved, or every recipient waits for approval. **Runs succeed but nobody is notified.** Check the status: **No results**, **Below threshold** and **Silenced** notify nobody by design. Then check the recipients and the delivery status in [Notifications](https://moberghr.github.io/beacon/features/notifications/). **Runs time out.** Narrow or index the query, or raise the subscription's timeout up to the deployment's limit. ## Related - [Queries](https://moberghr.github.io/beacon/features/queries/): the SQL a subscription runs - [Notifications](https://moberghr.github.io/beacon/features/notifications/): channels, recipients and delivery history - [Tasks](https://moberghr.github.io/beacon/features/tasks/): work items opened by **Create a task** - [Anomaly Detection](https://moberghr.github.io/beacon/features/anomaly-detection/): the Anomaly step - [Control Tower](https://moberghr.github.io/beacon/features/control-tower/): health across all subscriptions --- # Notifications > Deliver subscription results through five channels (Email, Microsoft Teams, Slack, Jira and a generic webhook) to vetted, encrypted recipients, with a delivery history. Source: https://moberghr.github.io/beacon/features/notifications/ When a subscription run [notifies](https://moberghr.github.io/beacon/features/subscriptions/#which-runs-notify), Beacon sends the result to each of the subscription's **recipients**. A recipient is one destination on one channel: an email address, a Slack or Teams webhook, a Jira project, or any HTTPS endpoint. Data-quality alerts use the same recipients. ## Channels | Channel | What Beacon sends | Inline rows | Attachment | |---|---|---|---| | **Email** | An HTML email through the host's `IEmailAdapter` | The preview rows, as an HTML table | CSV or XLSX, when the subscription includes one | | **Teams** | An Adaptive Card (1.5) posted to a Teams webhook | First 10 rows, first 3 columns | None | | **Slack** | A Block Kit message posted to an incoming webhook, the rows as a fixed-width text table in a code block | First 25 rows, first 5 columns, each column cut to 20 characters | None | | **Jira** | An issue (type Task) or a comment on it, through the Jira Cloud REST API | The preview rows, as a table | None | | **Webhook** | A JSON `POST` to any allowed HTTPS URL | The preview rows, as JSON | None | What every channel gets from a run: - **Title.** `[Beacon] {data source} - {query name}` (Email subject, Slack header; the Teams card shows the data source and query name). - **Preview rows.** A short preview of the result's first rows (at most 20), cut further by the subscription's **Max rows**, and then by the channel's own limits above. The total row count is always stated. - **Query text**, only when the subscription has **Show query text in notification** on. - **Anomaly details** (Slack and webhook), when [anomaly detection](https://moberghr.github.io/beacon/features/anomaly-detection/) flagged the run. - **A link back to Beacon**, `{BaseUrl}/notifications/{id}`; see [Links back to Beacon](#links-back-to-beacon). Rows are [PII-masked](https://moberghr.github.io/beacon/features/subscriptions/#what-a-run-reads-and-delivers) before they reach any channel, the attachment included. ### Attachments A subscription with **Include results as attachment** builds one file per run, **CSV** or **XLSX** as its **Attachment format** says. The file holds the result's rows up to the subscription's **Max rows** (and never more than `Beacon:Query:MaxExecutionRows`, since a larger result fails the run), PII-masked like every other row. Only email delivers it: Teams, Slack and webhooks ignore it, and Jira logs a warning and posts its comment without it. ### Links back to Beacon Messages link to the run's notification page as `{BaseUrl}/notifications/{id}` (Teams **View Query Results**, Slack **View Full Results**, the webhook's `notificationUrl`). Without a base URL, messages carry no link. The React app is served under `/beacon`, so **the base URL must include `/beacon`**: `https://beacon.example.com/beacon`, not `https://beacon.example.com`. The host sets it in code: ```csharp builder.Services.AddBeaconServices(builder.Configuration, options => { options.BaseUrl = builder.Configuration["Beacon:BaseUrl"]; // e.g. https://beacon.example.com/beacon }); ``` An Admin can change it at runtime under **Admin Settings** → **Base URL** (see [Admin Settings](https://moberghr.github.io/beacon/security/admin-settings/)). ## Recipients Recipients are managed on the **Recipients** page (`/beacon/recipients`). Only an Admin or a **SecurityAdmin** creates, changes or deletes them (`POST`, `PUT`, `DELETE /beacon/api/recipients`). Everyone else with Write on a subscription or data contract can attach existing recipients to it (an API key also needs the Execute scope), but never sees their destinations; Admins and SecurityAdmins see them masked. A SecurityAdmin changes where a recipient delivers (its type, destination or headers) only when it can read every subscription and data contract the recipient is attached to (`403` otherwise); renaming is always allowed. Click **Add recipient** and fill in: | Field | What it holds | |---|---| | **Notification type** | Email, Slack, Teams, Jira or Webhook. | | **Name** | How the recipient is listed and picked. | | Destination | Depends on the type; see the table below. | | **Description** | Optional. | | **Custom headers (JSON)** | Webhook only: a JSON object of HTTP headers (see [Webhook](#webhook)). | | **Body template** | Webhook only: a custom request body (see [Body templates](#body-templates)). | | Type | Destination field | Example | |---|---|---| | Email | **Email address** | `ops@example.com` (a bare address, no display name) | | Slack | **Slack webhook URL** | `https://hooks.slack.com/services/T…/B…/…` | | Teams | **Teams webhook URL** | A Teams Workflows webhook URL (see [Microsoft Teams](#microsoft-teams)) | | Jira | **Jira connection string** | `example;OPS;bot@example.com;` | | Webhook | **Webhook URL** | `https://hooks.example.com/beacon` | There is no test button: create the recipient, attach it to a subscription, and use **Test now** on the subscription to send a delivery. Destinations and header values are secrets: they are encrypted when saved, shown masked, and never returned by the API in full. Recipients saved before Beacon encrypted them keep working until re-saved or encrypted with `IRecipientSecretEncryptionService`, unless the host sets `Beacon:Notifications:RequireEncryptedSecrets`. A recipient whose stored destination cannot be decrypted (saved under another encryption key) is marked **Re-enter destination**; open it and enter the destination again. ### Destination policy Every destination is checked when it is saved and again when Beacon connects (`Beacon:Notifications`, see [Notification Destinations](https://moberghr.github.io/beacon/getting-started/configuration/#notification-destinations-optional)): - **HTTPS only**, and no user name or password in the URL. - **Vendor hosts** for the vendor channels: Slack `hooks.slack.com`; Teams `*.webhook.office.com`, `*.logic.azure.com` and `*.environment.api.powerplatform.com`; Jira `*.atlassian.net` and `api.atlassian.com`. `AllowedHosts:{Type}` extends a list. - **Webhooks** may use any public HTTPS host, unless `AllowedHosts:Webhook` is set, which then is the whole list. - **No private networks.** Private, loopback and link-local addresses are unreachable unless listed in `AllowedPrivateNetworks:{Webhook|Slack|Teams|Jira}`; Beacon connects only to an address it has checked. - **Email** takes bare addresses; `AllowedHosts:Email` limits the domains. - `DisabledTypes` turns channels off; `UseSystemProxy` sends deliveries through the system proxy, which then decides where the name resolves. A failed delivery shows a generic reason in the history, never the destination's response; the details are in the server log. ### Approval of new and changed destinations This applies only while the [approval workflow](https://moberghr.github.io/beacon/security/approvals/) is on (it is off by default; a host turns it on in code with `options.EnableApprovalWorkflow(...)`) and covers recipients (`CoversRecipientDestinations`, on by default once the workflow is on). Then where a recipient delivers is a reviewed change: - **A recipient created, or whose type, destination or custom headers change, receives nothing until a second person approves exactly where it now delivers.** The save is stored and submits a change request (the saver is the requester); the dialog says *"Submitted for approval: the recipient receives no deliveries until a second person approves where it delivers."* An Approver (or an Admin) other than the requester approves it on the **Approvals** page under **Recipients**. A recipient changed again since the reviewer opened the request is not approved (`409`). - **Deliveries skip a recipient that waits.** Every other recipient is still notified; the skipped one is logged as a Warning with ids only, and the recipients list and the subscription's **Recipients** tab mark it **Awaiting approval**. - **Saves that change nothing where it delivers** (name, description, body template) submit nothing. Putting back the approved destination needs no approval and withdraws the requester's own pending change. A change made while another person's change of the same recipient waits is refused (`409`). - **A rejected change** leaves the recipient skipped until it is changed back to its approved destination or saved again; the recipients list and the **Recipients** tab show **Rejected: change it and save again** with the reviewer's comment. - **Attaching an existing recipient** to a subscription or data contract is not reviewed. - **Rotating a credential** (a new webhook token, a new header value) is a change of where it delivers and waits for approval like any other. Recipients that existed before the workflow was turned on are not governed and keep delivering. Turning the workflow (or `CoversRecipientDestinations`) off again lets waiting and rejected recipients receive deliveries as they are; startup warns with their count. #### What reviewers see A reviewer sees where data will go without seeing any credential. The change request shows, before and after: - the **type**, and what **identifies the destination**: a webhook's scheme, host and port, with each path segment masked (so the path's shape is visible) and user info, query values and a fragment masked. Only the parts a provider documents as not secret are shown: Slack's workspace and webhook ids (the token masked); a Teams workflow's fixed path words and workflow id (its signature masked); a legacy Teams connector's group and tenant ids. Email addresses are shown as they are; a Jira destination shows its site, project and email; - the **custom header names**, each with a **fingerprint** of its value (12 hex characters), never the value; - a **fingerprint of the whole definition** (type, destination and headers). A fingerprint is a keyed digest (an HMAC under a key derived from `Beacon:EncryptionKey`): equal fingerprints mean equal values, and without the key it reveals nothing. Admins and SecurityAdmins see the same fingerprints on the **Recipients** page, so requester and reviewer can compare them out of band before the approval. When the before and after look the same but the fingerprints differ, the change lies only in a masked part (a new path, token, signature or header value). The approval page says so, and **approving it requires a comment** saying how the reviewer checked the change (`400` without one). Recipient requests are visible to Approvers and Admins only: recipients belong to no project. #### When a delivery is checked The check sits in `INotificationService.SendNotification`, the single path to every channel: the copy of the destination and headers about to be sent is decrypted, hashed and compared with the recipient's approved hash read at that moment. A recipient changed or archived after a run loaded it is therefore never sent the unapproved copy, and a recipient that no longer exists receives nothing. A host that calls `SendNotification` itself with a `RecipientId` gets the same check; a delivery without a `RecipientId` is not a governed recipient's and is not checked. The approved hash is an HMAC-SHA256 under a key derived from `Beacon:EncryptionKey`, never a plain hash of the destination. A hash written under a key listed in `Beacon:PreviousEncryptionKeys` still matches, and the re-encryption run (`IStoredValueReEncryptionService`, see [Configuration → Replacing the Key](https://moberghr.github.io/beacon/getting-started/configuration/#replacing-the-key)) recomputes the approved hashes of governed recipients and their pending requests under the current key. A hash it cannot recompute (an approved destination the recipient has since moved away from while a change waits) is reported as an unresolved key dependency: decide the pending change, or save the recipient back to its approved destination, then run the re-encryption again before removing the previous key. #### Runs that notified nobody A run whose every recipient waits for approval notifies nobody: it is recorded as **Skipped** with *"Skipped: every recipient waits for approval of where it delivers, so nobody was notified. Approve the recipients' changes on the Approvals page."*, a Warning is logged, and **Test now** answers `409` with that reason. A run that notified some recipients names each skipped one in its comment. A skipped run is not a baseline for the result-count trigger or for anomaly detection. ## Email Beacon does not send email itself: the host implements `IEmailAdapter` and registers it. ```csharp using Beacon.Core.Adapters; using Beacon.Core.Adapters.Mail; public sealed class SmtpEmailAdapter : IEmailAdapter { public async Task SendEmailAsync(string to, string subject, string body, QueryResultFile? queryResultAttachmentFile) { // body is HTML. queryResultAttachmentFile, when present, has Name, ContentType and Data (bytes). // Send with your provider (SMTP, SendGrid, Amazon SES, ...). } } ``` ```csharp builder.Services.AddBeaconServices(builder.Configuration, options => { options.AddEmailAdapter(); }); ``` A destination may list several addresses separated by commas or semicolons; a message to more than 50 addresses is refused (the run records a failed delivery). `IEmailAdapter` also has an overload that takes a `CancellationToken`; Beacon calls it, and by default it forwards to the overload above and stops waiting when the token is cancelled. Override it when your sender can cancel the send itself. Register an adapter before you create email recipients. The email has the subject `[Beacon] {data source} - {query name}`, and an HTML body with the query text (when shown), the total row count and the preview rows as a table; every value in it is HTML-encoded. See also [Configuration → Email Adapter](https://moberghr.github.io/beacon/getting-started/configuration/#email-adapter). ## Microsoft Teams Office 365 connector URLs on `outlook.office.com` are retired and refused by the default destination policy. Use a Teams **Workflows** webhook: in Teams, create a workflow (Power Automate) that starts when a webhook request is received and posts the card it receives to your channel, then copy its URL. Workflow URLs are on `*.logic.azure.com` or `*.environment.api.powerplatform.com`; legacy `*.webhook.office.com` URLs are still accepted. Beacon posts an Adaptive Card (version 1.5) as the request body: a "Beacon" title, `{data source}: {query name}`, the query text when shown, and a table of the first 10 rows and 3 columns (headed "First 10 records" when there are more), plus a **View Query Results** button when a [base URL](#links-back-to-beacon) is set. ## Slack Create a Slack app with **Incoming Webhooks** on, add a webhook to the channel, and copy its URL (`https://hooks.slack.com/services/T…/B…/…`). Anyone holding the URL can post to the channel. Beacon posts a Block Kit message: a header `[Beacon] {data source} - {query name}`, an anomaly block when the run was flagged, the query in a code block when shown, the line *Results: Showing X of Y total records*, the rows as a fixed-width text table inside a code block (at most 25 rows and 5 columns, each column cut to 20 characters), and a **View Full Results** button when a [base URL](#links-back-to-beacon) is set. ## Jira The destination is one string, **`domain;project-key;email;api-key`**: ```text example;OPS;beacon-bot@example.com;ATATT3xFfGF0... https://example.atlassian.net;OPS;beacon-bot@example.com;ATATT3xFfGF0... https://api.atlassian.com/ex/jira/;OPS;beacon-bot@example.com;ATATT3xFfGF0... ``` The email and API token are a Jira Cloud account's that may create issues in the project. Beacon uses the Jira Cloud REST API; an on-premises Jira is reachable only when the host allows its host (`AllowedHosts:Jira`) and network (`AllowedPrivateNetworks:Jira`). A delivery creates an issue of type **Task** in the project, with the summary `BEACON: {query name}`, the labels `Beacon` and `beacon-sub-{subscription id}-rcpt-{recipient id}` and priority **Medium**, its description holding the total row count, the query text when shown and the preview rows as a table; the same content is added as a comment. A later delivery adds a comment to that issue when Beacon finds it, and creates a new one otherwise. Beacon finds the issue by that second label among the project's open issues (a resolved issue is not reused: the next delivery with rows opens a new one). For this release an open issue created before the label existed is still matched by its `BEACON: {query name}` summary within the same project and reporter, but it never gets the label, so it is replaced by a labelled issue after it is resolved. Jira gets no attachment. ## Webhook A webhook recipient receives an HTTPS `POST` with a JSON body (`Content-Type: application/json`). By default the body is: ```json { "timestamp": "2026-10-15T09:00:04.512Z", "source": "Beacon", "subscriptionName": "Unpaid completed orders", "subscriptionId": 42, "dataSourceName": "Orders DB", "sqlQuery": "SELECT id, status, payment_status FROM orders WHERE ...", "totalRecords": 3, "executionTimeMs": 184.2, "previousResultCount": 1, "timedOut": false, "records": [ { "id": 1017, "status": "completed", "payment_status": "pending" } ], "anomaly": { "isAnomaly": true, "severity": "Medium", "explanation": "..." }, "notificationUrl": "https://beacon.example.com/beacon/notifications/981" } ``` | Field | What it holds | |---|---| | `timestamp` | When the message was built (UTC). | | `source` | Always `Beacon`. | | `subscriptionName`, `subscriptionId` | The query name the subscription is shown by, and the subscription id. | | `dataSourceName` | The data source the result came from. | | `sqlQuery` | The query text; present only when the subscription shows the query. | | `totalRecords` | The run's full row count. | | `executionTimeMs` | How long the run took. | | `previousResultCount` | The row count of the last run that was not skipped; absent on the first run. | | `timedOut` | Whether the run timed out. | | `records` | The preview rows (PII-masked), as objects keyed by column name. | | `anomaly` | `isAnomaly`, `severity`, `explanation`; present only when anomaly detection flagged the run. | | `notificationUrl` | The link back to Beacon; present only when a [base URL](#links-back-to-beacon) is set. | Fields without a value are left out. **Custom headers.** **Custom headers (JSON)** takes a JSON object of string values, at most 20 headers, each value at most 4,096 characters without control characters: ```json { "Authorization": "Bearer " } ``` Only these names are allowed: `Authorization`, `Api-Key`, `X-Api-Key` and `Ocp-Apim-Subscription-Key`, plus the names the host adds in `Beacon:Notifications:AllowedHeaders`. Headers that control the connection, routing, cookies or cloud metadata access (`Host`, `Content-Type`, `Cookie`, `X-Forwarded-For`, `Metadata-Flavor` and others) are never allowed. Header values are encrypted like the destination; when editing, a masked value keeps the stored one, and clearing the field removes all headers. ### Body templates A recipient's **Body template** replaces the default message body. The dialog offers it for webhooks; the Teams and Slack channels honour one too when it is set through the API (`bodyTemplate` on `POST`/`PUT /beacon/api/recipients`), where the template must produce that channel's own JSON. Placeholders are written `{{Name}}`, case-insensitive: `{{Timestamp}}`, `{{Source}}`, `{{SubscriptionName}}`, `{{SubscriptionId}}`, `{{DataSourceName}}`, `{{SqlQuery}}`, `{{TotalRecords}}`, `{{ExecutionTimeMs}}`, `{{PreviousResultCount}}`, `{{TimedOut}}`, `{{Records}}`, `{{NotificationUrl}}`, and `{{Anomaly.IsAnomaly}}`, `{{Anomaly.Severity}}`, `{{Anomaly.Explanation}}`. - `{{Records}}` becomes compact JSON; add `_Pretty` to a placeholder (`{{Records_Pretty}}`) for indented JSON. - Dates are ISO 8601, booleans `true`/`false`, decimals two places; a missing value is empty. - The template is read once: a value that itself contains `{{...}}` is inserted as text, never expanded again. Text values are escaped for JSON (quotes, backslashes, control characters), so write `"{{SubscriptionName}}"` inside quotes; `{{Records}}` and `_Pretty` placeholders stay raw JSON. An expansion over 1 MB fails and the default message is sent. - In a **Slack** template, text values (also every string inside `{{Records}}`) are escaped the way Slack documents it: `&`, `<` and `>` become `&`, `<` and `>`, so a value cannot form a mention, link or command. In a **Teams** template, `*`, `_`, `` ` ``, `~`, `[`, `]`, `<` and `\` in a value are backslash-escaped so it cannot form emphasis, links or tags. Direction-override, zero-width and control characters are removed from values. - An unknown placeholder is left as `{{UNKNOWN:Name}}`. If the template cannot be processed, Beacon sends the default body. ```json { "text": "{{SubscriptionName}}: {{TotalRecords}} rows", "rows": {{Records}}, "link": "{{NotificationUrl}}" } ``` ## Notification history **Notifications** (`/beacon/notifications`) lists subscription runs: **When**, **Query**, **Recipients**, **Rows** and **Status**, filtered by status and paged on the server (20 rows per page, at most 200). The statuses are those of [Which runs notify](https://moberghr.github.io/beacon/features/subscriptions/#which-runs-notify): Created, Sent, Silenced, No results, Timeout, Below threshold, Failed and Skipped. A notification's page (`/beacon/notifications/{id}`) shows the delivery (type, created and sent times), the recipient, the query and subscription, the execution time and result count, and the **Stored results** when the subscription stores result rows. A failed run's comment is shown only when it is a recorded failure reason (for example `Recipient 7: Notification delivery failed: the destination did not respond in time.`); any other stored text is shown as a generic reason, and the details are in the server log. ### Who sees a run The notification list and pages show stored runs, including their result rows; the Home activity feed, the Control Tower detail, a task's executions and result history and a subscription's anomaly chart list the same runs. Each run records the data sources it read, and a run is visible: - to an **Admin**, always; - to a **signed-in user**, when one project they may read holds every data source the run read, or when every data source it read is openly reachable; - to an **API key or MCP caller restricted to projects**, only when one of its projects holds every data source the run read (by the projects' current data sources). Runs recorded before Beacon kept their data sources are hidden from such callers; - to an API key or MCP caller with no project, or an anonymous request: never. Runs you may not see are left out of lists, and their pages read as not found. ### Delivery failures Every recipient of a run is tried; one failing does not stop the others. A run with any failed delivery is recorded as **Failed**, naming the failed recipient ids with a generic reason. Beacon does not retry a failed delivery: fix the recipient, then use **Test now** on the subscription or wait for the next run. ### Real-time updates The React app receives updates over a SignalR hub at `/beacon/api/hub`: `JobStatusChanged` and `ApprovalUpdated`, each scoped to the current user. A host that does not want them sets `AddBeaconApiServices(x => x.Realtime = false)`, and the app falls back to manual refresh. ## Troubleshooting **Email does not arrive.** Check that the host registered an `IEmailAdapter` (`options.AddEmailAdapter()`), the adapter's own logs and provider settings, the address, and the spam folder. **Saving a recipient is refused.** The destination is outside the [destination policy](#destination-policy): not HTTPS, not a vendor host, a private address, a display name in an email address, or a header that is not allowed. The message says which. **Teams delivers nothing.** Use a Workflows URL; `outlook.office.com` connector URLs are refused. Check that the workflow posts the card it receives to the channel. **Jira delivers nothing.** Check the four parts of the connection string, that the token's account may create issues in the project, and that the site is on `*.atlassian.net` (or allowed by the host). **The link in a message leads to a blank or wrong page.** The base URL is missing `/beacon`; see [Links back to Beacon](#links-back-to-beacon). **A recipient shows Awaiting approval.** Its destination waits for a second person's approval on the **Approvals** page. ## Related - [Subscriptions](https://moberghr.github.io/beacon/features/subscriptions/): which runs notify and what they carry - [Approvals](https://moberghr.github.io/beacon/security/approvals/): the change-review workflow - [Configuration → Notification Destinations](https://moberghr.github.io/beacon/getting-started/configuration/#notification-destinations-optional): the `Beacon:Notifications` settings - [Data Quality](https://moberghr.github.io/beacon/features/data-quality/): contract alerts use the same recipients ## What the default Slack, Teams and email messages do with values Row values, column names, SQL, names and anomaly text are encoded for the channel (Slack `& < >` escaping and no way to close a code fence; Teams markdown escaping; HTML encoding in email), and direction-override, zero-width and other invisible format characters are removed. Messages stay within the channel's limits: a Slack section is kept under 3,000 characters, a Slack header under 150, and a Teams card under about 24 KB; what does not fit is cut and marked `(truncated)`, and rows that do not fit are left out. The email subject is one line of at most 200 characters. --- # Control Tower > Health of every subscription you can read: health buckets, success rates, open tasks and anomaly sparklines, refreshed every 30 seconds. Source: https://moberghr.github.io/beacon/features/control-tower/ The Control Tower is Beacon's operations view. It shows the health of every [subscription](https://moberghr.github.io/beacon/features/subscriptions/) you can read over a time window: success rate, runs and failures, open tasks and anomaly activity, so you can spot failing or silent monitors early. ![Control Tower](https://moberghr.github.io/beacon/img/screenshots/control-tower-dark.png) Open it from **Control Tower** in the navigation (`/beacon/control-tower`). ## What you see The page answers three questions per subscription: - **Is it working?** Success rate, run and failure counts, and a health status (Green, Amber, Red, Stalled). - **Does it need attention?** The number of open (unresolved) [tasks](https://moberghr.github.io/beacon/features/tasks/). - **Is the data behaving?** Anomaly counts and a per-day sparkline from [anomaly detection](https://moberghr.github.io/beacon/features/anomaly-detection/). Everything on the page (the KPI cards, the rows and the detail panel) covers only subscriptions in projects you can read. A project-restricted API key or MCP caller sees only the runs its projects may read. ## Health statuses Each subscription gets one status, computed from its runs **within the selected time window** (default 30 days). | Status | Rule | |--------|------| | **Green** | Success rate of at least 90% | | **Amber** | Success rate of at least 70% and below 90% | | **Red** | Success rate below 70% | | **Stalled** | No runs in the window, and the subscription was created more than 1 day ago | - **New subscriptions stay Green.** A subscription with no runs that was created within the last day counts as Green (waiting for its first run), not Stalled. - **Stalled ignores the past.** A subscription that ran well last quarter but has no runs in the selected window is Stalled. Silence is a failure mode. ### What counts as a successful run The success rate is `successful runs / total runs × 100`. A run is successful when its notification status is one of: - `NotificationSent`: the query ran and a notification went out; - `NoResults`: the query ran and returned nothing to alert on; - `NotificationSilenced`: the query ran and the notification was silenced (for example, no anomaly was detected). Every other outcome (`Failed`, `Timeout`, `Created`, `BelowThreshold`, `Skipped`) counts against the subscription. The failed count shown is `total − successful`. A `Skipped` run is a scheduled run that did not execute: either the subscription's owner (the user its scheduled runs run as) may no longer run its query, or the query's change is not approved. Skipped runs count as unsuccessful, so a subscription whose runs keep being skipped turns Amber and then Red. ## KPI cards Six cards sit at the top, scoped to the current filters and time window: | Card | Value | |------|-------| | **Total** | Subscriptions matching the filters | | **Healthy** | Green subscriptions; the subtitle is the overall success rate (all successes ÷ all runs) | | **Warning** | Amber subscriptions | | **Critical** | Red subscriptions | | **Stalled** | Subscriptions with no runs in the window (created more than a day ago) | | **Open tasks** | Unresolved tasks across matching subscriptions; the subtitle is the anomaly count in the window | When data contracts you can read are not evaluated because their owner may no longer run them, a warning under the cards gives the count and links to [Data Quality](https://moberghr.github.io/beacon/features/data-quality/). Statistics are cached on the server for 30 seconds per filter combination. With no runs at all, the overall success rate is 100%. ## The health table Each matching subscription is a row: | Column | Content | |--------|---------| | (status) | Green, Amber, Red or Stalled pill | | **Subscription** | Query name and folder path, with badges: **AI · {actor}** when an AI actor is assigned, **Anomaly** when anomaly detection is on, **Owner not authorized** when the owner may no longer run the query, and **Not approved** when the last run was skipped because the query's change is not approved | | **Success** | Success rate in the window (n/a when there are no runs) | | **Runs** | Total runs, with the failed count highlighted when above zero | | **Tasks** | Open task count as a warning pill, or the total task count when nothing is open | | **Anomalies** | Sparkline of anomaly events per day across the window, with the total | | **Last run** | Outcome pill (Sent, Failed, Timeout, No results, Silenced, Skipped, …) and the time of the most recent run | The table shows **20 rows per page**. Filters, sort and page are kept in the URL. ### Sorting Without a sort the table is **worst first**: most open tasks first, then lowest success rate. Click a column header to sort by **Subscription** (query name), **Success**, **Runs** or **Tasks** instead. ### Row detail Click a row to open its detail panel: - the query name and the cron schedule, with **Open subscription** and, when tasks are open, a link to them; - a **Not approved** notice when the last run was skipped because the query's change is not approved (save the query to submit its SQL for approval, or restore a version and approve it; see [Approvals](https://moberghr.github.io/beacon/security/approvals/)); - an **Owner not authorized** notice when scheduled runs are skipped because of the owner; an Admin or a UserAdmin assigns another owner on the subscription page (see [Subscriptions](https://moberghr.github.io/beacon/features/subscriptions/)); - **recent executions**: the last 20 runs with outcome, time, row count, execution time and, for failed runs, the recorded failure reason (other stored text shows as a generic reason); - **open tasks**: up to 20 unresolved tasks with priority, assignee and snooze state; - **recent anomalies**: up to 20 anomaly events in the window with severity, detected value, explanation and acknowledged state. ## Filters All filters combine and apply to both the KPI cards and the table: | Filter | Options | What it does | |--------|---------|--------------| | Search | Free text (applied on Enter or when the field loses focus) | Matches the query name | | Folder | Any query folder | Only subscriptions whose query is in that folder | | Health | All, Green, Amber, Red, Stalled | Only subscriptions in that status | | Time range | 24h, 7d, 30d, 90d | The window for every count, rate, status and sparkline (default 30d) | | **open tasks only** | Checkbox | Only subscriptions with at least one unresolved task | Changing a filter returns to the first page. **Clear** resets the filters. :::tip The window changes what healthy means. A subscription that failed once yesterday can be Red over 24h and Green over 30d. Use a short window for incident triage and a long one for trend review. ::: ## Refresh - The page refetches every 30 seconds and when the browser tab regains focus. - **Refresh** reloads at once. - If a selected subscription drops out of the refreshed result (for example, it no longer matches the filters), its detail panel closes. ## REST API Three read-only routes under `/beacon/api/control-tower`: | Route | What it returns | |-------|-----------------| | `GET /statistics` | KPI counts for the filters, as `{ "statistics": { … } }` | | `GET /health` | A page of per-subscription health rows | | `GET /subscriptions/{id}/detail` | Recent runs, open tasks and anomalies of one subscription (`timeRangeDays`, default 30) | `/statistics` and `/health` take the same filters: | Parameter | Type | Default | What it does | |-----------|------|---------|--------------| | `timeRangeDays` | integer | `30` | Size of the window in days | | `folderId` | integer | | One query folder | | `dataSourceId` | integer | | One data source | | `healthStatus` | enum | | `Green`, `Amber`, `Red` or `Stalled` | | `hasUnresolvedTasks` | boolean | | `true` for subscriptions with open tasks only | | `searchKeyword` | string | | Substring of the query name | `/health` is a paged list and also takes: | Parameter | Default | What it does | |-----------|---------|--------------| | `page` | `0` | Zero-based page number | | `pageSize` | `20` | Rows per page; values above 200 are lowered to 200 | | `sort` | worst first | Comma-separated columns, `-` prefix for descending: `queryName`, `successRate`, `totalExecutions`, `unresolvedTaskCount` | Example: ```bash curl "https://your-beacon-host/beacon/api/control-tower/health?timeRangeDays=7&healthStatus=Red&sort=-totalExecutions" \ --cookie "Beacon.Auth=..." ``` ```bash curl "https://your-beacon-host/beacon/api/control-tower/statistics?timeRangeDays=7" \ --cookie "Beacon.Auth=..." ``` The statistics object has `totalSubscriptions`, `healthySubscriptions`, `warningSubscriptions`, `criticalSubscriptions`, `stalledSubscriptions`, `totalUnresolvedTasks`, `totalAnomalies30Days` (the anomaly count in the requested window), `overallSuccessRate`, `timeRangeDays` and `ownerNotAuthorizedContracts`. ## Suggested workflow 1. **Keep the default sort.** Worst first puts subscriptions with open tasks and low success rates on top. 2. **Triage Red and Stalled first.** Red means runs fail; Stalled means nothing runs. Open the row and read the failure reason in the recent executions. **Owner not authorized** needs a new owner; **Not approved** needs the query change approved. 3. **Use open tasks only for follow-up.** After an incident, filter to subscriptions with open tasks and work the list down. 4. **Watch the sparklines.** A rising anomaly sparkline on a Green subscription means the runs succeed but the data is drifting. 5. **Widen the window for reviews.** Use 90d in a periodic review to catch slow degradation. ## Related - [Subscriptions](https://moberghr.github.io/beacon/features/subscriptions/): the scheduled runs the Control Tower monitors - [Tasks](https://moberghr.github.io/beacon/features/tasks/): the open tasks in the table and detail panel - [Anomaly Detection](https://moberghr.github.io/beacon/features/anomaly-detection/): the source of anomaly counts and sparklines - [Notifications](https://moberghr.github.io/beacon/features/notifications/): the delivery outcomes behind the success rate --- # Tasks > Work items that open when a subscription's query returns rows, stay open while it keeps returning rows, and resolve themselves when it returns none. Source: https://moberghr.github.io/beacon/features/tasks/ A task tracks an alert condition of one subscription over time. When a run of a subscription with task creation on returns rows, Beacon opens a task. Later runs update it, and a run that returns no rows resolves it. In between, people claim it, set its priority, snooze it and log what they find. ![Tasks](https://moberghr.github.io/beacon/img/screenshots/tasks-dark.png) - **One open task per subscription.** Later runs update the open task instead of creating another. - **Resolves itself.** A run that returns 0 rows resolves the open task. - **Context.** A task links to its subscription, query, runs and notifications. - **Team workflow.** Priority, assignee, watchers, snooze, an SLA indicator and an investigation log. [Notifications](https://moberghr.github.io/beacon/features/notifications/) say that something happened; tasks track whether someone dealt with it. ## Turn on task creation Task creation is set when you create the subscription; a subscription cannot be edited afterwards. 1. In **Subscriptions** (`/beacon/subscriptions`), click **New subscription**. 2. Tick **Create a task and keep it open while the query returns rows**. 3. Finish and save the subscription. With task creation on, recipients are optional: a subscription can alert through the task list alone. Without it, at least one recipient is required. See [Subscriptions](https://moberghr.github.io/beacon/features/subscriptions/). ## Task lifecycle Every run of the subscription that executes its query (scheduled or manual) updates the task, whatever the notification status of the run: | Run result | Open task? | What happens | |------------|------------|--------------| | more than 0 rows | No | A new open task is created with the run's row count. | | more than 0 rows | Yes | The open task's latest row count and timestamps are updated. | | 0 rows | No | Nothing. | | 0 rows | Yes | The task is resolved automatically. | ### Automatic resolution When a run returns 0 rows while a task is open, Beacon resolves it at the time of the run with the note: ```text Auto-resolved: Query returned 0 results ``` An automatically resolved task records no resolver and keeps its assignee. It stays in the history with its note. :::caution Automatic resolution keys off a run with 0 rows. If your query can return 0 rows while the problem persists (a short time window during a quiet period, for example), the task closes and a new one opens when rows come back. Choose the query's window with that in mind. ::: ### Resolve a task A task is resolved by its **assignee**: claim it first (assign it to yourself), then resolve it. An Admin may also resolve a task that **nobody** is assigned to; to resolve a task assigned to someone else, an Admin first reassigns it to themselves. 1. Open the task and click **Resolve** (or press `R`). 2. Optionally add resolution notes (up to 2,000 characters). 3. Confirm. The task records who resolved it, when, and the notes. **Resolve** is offered only to those who may resolve; the server refuses others with `403` and says to claim the task first. A resolution is never overwritten: resolving a resolved task fails. If the condition comes back on a later run, a **new** task opens; a resolved task is not reopened, so each incident keeps its own history. ## What a task records | Field | What it is | |-------|------------| | Latest result count | Row count of the most recent run | | Executions | Runs of the subscription since the task was created | | Unique counts | Distinct row counts seen since the task was created, a volatility signal | | Notifications | Notifications sent while the task was open | | Priority | `Critical`, `High`, `Normal` (default) or `Low`, shown as P1 to P4 | | Assignee | The user working the task (none by default) | | Snoozed until | While in the future, the task is snoozed | | Watchers | Users following the task | | Resolution | Resolved flag, time, resolving user and notes | A task with 40 executions and 1 unique count returned the same row count every time: a stable condition. One with 40 executions and 25 unique counts changes on almost every run: a moving or growing problem. ## Tasks page Open **Tasks** in the navigation (`/beacon/tasks`). The list shows only tasks of subscriptions in projects you can read. - **Columns:** **Id**, **Created**, **Subscription** (with the query name), **Latest**, **Execs**, **Unique** and **Status** (**Unresolved** or **Resolved**). Click a header to sort. - **Filter:** **Unresolved** (default), **Resolved** or **All**. - **Paging:** 20 tasks per page, on the server. Filters, sort and page are kept in the URL. Click a row to open the task. ## Task detail The task page (`/beacon/tasks/{id}`) shows: - **Header:** the task, its age and the **Resolve** action. - **SLA banner:** time left against the subscription's SLA hours, or 24 hours when the subscription sets none. For a resolved task it shows who resolved it and when. - **KPIs:** latest row count, executions, notifications and age. - **Result trend:** the row counts of the subscription's last 100 runs. - **Tabs:** | Tab | Contents | |-----|----------| | **Activity** | Timeline of the task: creation, runs, comments, resolution | | **Executions** | The last 50 runs of the subscription with time, duration, notification status and row count | | **Notifications** | Notifications sent for the subscription while the task was open | | **Related** | Up to 20 other tasks from subscriptions on the same query, archived ones included | The result trend and the **Executions** tab list only runs you may read. A project-restricted API key or MCP caller sees the runs its projects may read, by the rule [notifications](https://moberghr.github.io/beacon/features/notifications/) follow. ### Investigation log Every task has a comment thread. Comments record author and time, show newest first, and stay after resolution, so the next person who meets the same condition can read them (the **Related** tab leads there). ### Assign, watch, snooze, priority | Action | Who may do it | |--------|---------------| | **Assign** | Anyone who can edit may assign an unassigned task to themselves. The assignee may hand it over or release it. Assigning to someone else, or reassigning a task assigned to someone else, is an Admin's. | | **Watch / Unwatch** | Follow a task you are not assigned to; the watcher count shows on the task. | | **Resolve** | The assignee; an Admin only while nobody is assigned. | | **Snooze** | The assignee or an Admin. Not on a resolved task. | | **Priority** | The assignee or an Admin. Not on a resolved task. | With Beacon user management, the assignee must be an existing, enabled user. A host without it accepts the id its sessions present. Resolving, assigning, snoozing and setting the priority are written only while the task still has the state the check saw. If someone resolves or reassigns it in between, the request fails (`400`) and nothing is overwritten. ### Keyboard shortcuts On the task page, when you are not typing in a field: | Key | Action | |-----|--------| | `R` | Open the Resolve dialog | | `A` | Assign the task to yourself | | `S` | Snooze for 1 hour | | `C` | Focus the comment box | ## How people are identified Tasks identify people by their user's external id (`Users.ExternalId`), however they sign in. An API key acts as its owner. In a host without Beacon user management, the id is the `NameIdentifier` the session presents. The task detail's `assignedToCaller` says whether the signed-in caller is the assignee. For a task whose subscription an AI actor manages, the detail names the actor's creator by display name, or as "User {id}" (`hasOwner`, `ownerDisplayName`), never by user name or external id. "Admin" means a user holding the Admin role; a disabled or archived user is never an Admin. To anyone but an Admin, a task that does not exist is refused (`403`) like a task that is not theirs. ## REST API Routes under `/beacon/api` (the paths below start at `/tasks`). A scoped API key needs the Execute scope for the `POST` routes. | Method | Route | What it does | |--------|-------|--------------| | `GET` | `/tasks` | List tasks. Filters `subscriptionId`, `resolved`; paging `page` (zero-based), `pageSize` (default 20, at most 200), `sort` (for example `-id`). | | `GET` | `/tasks/{id}` | Task detail. | | `POST` | `/tasks/{id}/resolve` | Resolve, with optional `resolutionNotes`. The resolver is the caller. | | `GET` | `/tasks/{id}/executions` | Recent runs of the task's subscription (the runs the caller may read). | | `GET` | `/tasks/{id}/related` | Tasks from other subscriptions on the same query. | | `GET` | `/tasks/{id}/result-history` | Row counts for the trend chart (the runs the caller may read). | | `GET` | `/tasks/{id}/comments` | Comment thread. | | `POST` | `/tasks/{id}/comments` | Add a comment. | | `POST` | `/tasks/{id}/assign` | Set or clear the assignee. | | `POST` | `/tasks/{id}/snooze` | Set or clear the snooze time. | | `POST` | `/tasks/{id}/priority` | Set the priority. | | `POST` | `/tasks/{id}/watch`, `/tasks/{id}/unwatch` | Follow or stop following the task. | :::note[For host developers] `ITaskService.ResolveTask` checks nothing about the caller: a host calling it in-process decides who may resolve. It throws `InvalidOperationException` for a task that is already resolved and `BeaconException` for a missing one. `ITaskService.GetTaskExecutionHistory` and `GetResultCountHistory` take a `StoredRunScope`. ::: ## Example: failed payments **Subscription** - Query: `SELECT * FROM payments WHERE status = 'failed' AND created_at > NOW() - INTERVAL '1 hour'` - Cron: `*/15 * * * *` (every 15 minutes) - **Create a task and keep it open while the query returns rows**: ticked **What happens** 1. A run finds 12 failed payments: a task opens with latest result count 12. 2. The next runs find 12, 15, 18: the same task updates; executions climb and unique counts show it is growing. 3. An engineer presses `A` to claim it, sets the priority to High and logs findings. 4. After the payment provider recovers, a run returns 0 rows: the task resolves with `Auto-resolved: Query returned 0 results`. 5. If failures come back next week, a new task opens, and the old one is in the **Related** tab. ## Troubleshooting | Symptom | Cause | |---------|-------| | No task after the subscription ran | Task creation is off on the subscription, or the run returned 0 rows (a 0-row run never opens a task). | | No second task while one is open | Expected: at most one open task per subscription; the open task is updated. | | The task closed on its own | A run returned 0 rows. The resolution note reads `Auto-resolved: Query returned 0 results`. | | The execution count looks low | It counts runs since the task was created, not since the subscription was created. | | **Resolve** is missing | You are not the assignee. Claim the task first (`A`), or ask an Admin. | ## Related - [Subscriptions](https://moberghr.github.io/beacon/features/subscriptions/): the scheduled queries that create tasks - [Control Tower](https://moberghr.github.io/beacon/features/control-tower/): open task counts across subscriptions - [Notifications](https://moberghr.github.io/beacon/features/notifications/): deliveries of the same runs - [Anomaly Detection](https://moberghr.github.io/beacon/features/anomaly-detection/): alert on unusual row counts instead of fixed conditions --- # Anomaly Detection > Statistical detection on a subscription's row count. Beacon learns a baseline from past runs and notifies only when a run deviates from it. Source: https://moberghr.github.io/beacon/features/anomaly-detection/ Anomaly detection changes when a [subscription](https://moberghr.github.io/beacon/features/subscriptions/) notifies. Instead of its notification trigger, Beacon compares each run's **row count** with the row counts of the subscription's earlier runs and sends the notification only when the run is unusual. It is statistical (standard deviation, interquartile range or percentage change), part of `Beacon.Core`, and needs no LLM. :::note[The metric is the row count] Beacon measures the number of rows a run returns, not a value inside the result. A query such as `SELECT COUNT(*) FROM …` always returns one row, so its row count never changes. Write the query to return one row per item you want to count, for example `SELECT id FROM orders WHERE created_at >= CURRENT_DATE`. ::: ## Turn it on Anomaly detection is set when you create the subscription; a subscription cannot be edited afterwards. 1. In **Subscriptions** (`/beacon/subscriptions`), click **New subscription**. 2. On the **Anomaly** step, tick **Enable anomaly detection**. 3. Set the fields below and finish the subscription. | Field | Default | Dialog accepts | What it does | |-------|---------|-------|--------------| | **Detection method** | Standard deviation | Standard deviation, IQR, Percentage change | How a run is compared with the baseline. See [Methods](#methods). | | **Sensitivity** | Medium | Low, Medium, High | The threshold of the method. High alerts more often. IQR ignores it. | | **Lookback (days)** | 30 | 7–365 | Age of the baseline runs: only runs from the last N days count. | | **Minimum data points** | 7 | 3–100 | Baseline runs needed before any run can be an anomaly. | | **Alert on an unusual increase** | on | | Alert when the row count is unusually high. | | **Alert on an unusual decrease** | on | | Alert when the row count is unusually low. The dialog requires at least one of the two. | The ranges and the either/or rule are the dialog's. Through the API, the same settings go in the `anomalyConfig` object of `POST /beacon/api/subscriptions` (`detectionMethod`, `sensitivity`, `lookbackDays`, `minimumDataPoints`, `alertOnIncrease`, `alertOnDecrease`), and the API stores the values it is given. ## How a run is evaluated For a subscription with anomaly detection on, each run that executes its query goes through these steps: 1. **Evaluate.** Beacon loads the stored baseline values of the subscription from the last **Lookback** days. With fewer than **Minimum data points** values, the run is not an anomaly ("Not enough historical data … Building baseline…"). Otherwise the configured method decides. 2. **Decide the notification.** The usual checks come first: a run that timed out is `Timeout`, a run with 0 rows is `NoResults`, and a run below the subscription's minimum row count is `BelowThreshold`. After those, an anomaly sends the notification (`NotificationSent`) and any other run is silenced (`NotificationSilenced`). The subscription's notification trigger is not used while anomaly detection is on. A run whose query returns no rows ends as `NoResults`; anomaly detection compares runs that return rows. The [Control Tower](https://moberghr.github.io/beacon/features/control-tower/) shows `NoResults` runs. 3. **Store the baseline.** The run's row count is stored as a new baseline value, with the time of the run. A run is not stored when all of its recipients wait for approval of their destination, so nobody could be told about it. 4. **Record the event.** For an anomaly, Beacon records an anomaly event (current value, baseline mean, standard deviation, z-score, severity, explanation), linked to the first notification that was delivered, if any. Recipients receive the evaluation with the notification; the webhook payload carries it as `Anomaly`. Baselines are stored only while anomaly detection is on for the subscription. Runs that did not execute the query (skipped because the owner may no longer run it, or because the query's change is not approved) store nothing. [Tasks](https://moberghr.github.io/beacon/features/tasks/) are independent of anomaly detection: they follow the **Create a task** option of the subscription and open on any run with rows. ## Methods All three methods use the baseline values of the lookback window, oldest to newest, without the current run. ### Standard deviation Beacon computes the mean (μ) and the population standard deviation (σ) of the baseline, then the z-score of the run: `z = (x − μ) / σ`. The run is an anomaly when `|z|` is above the sensitivity's threshold and its direction (above or below the mean) has alerting turned on. | Sensitivity | Threshold | |-------------|-----------| | High | 1.5σ | | Medium | 2σ | | Low | 3σ | Severity by `|z|`: Critical from 4, High from 3, Medium from 2, otherwise Low. When every baseline value is the same (σ = 0), the z-score is 0 and no run is an anomaly. ```text Baseline: [10, 12, 11, 13, 10, 12, 11] mean 11.29, σ 1.03 Run: 18 rows z = (18 − 11.29) / 1.03 = 6.5 Medium: 6.5 > 2 → anomaly, severity Critical ``` Suits steady data. ### IQR (interquartile range) Beacon sorts the baseline and takes Q1 and Q3 (the values at 25% and 75% of the sorted list), then `IQR = Q3 − Q1`. The run is an anomaly when it is below `Q1 − 1.5 × IQR` (with decrease alerts on) or above `Q3 + 1.5 × IQR` (with increase alerts on). The multiplier is fixed at 1.5; **Sensitivity** does not change it. Severity by how many IQRs the run lies beyond the bound: Critical from 3, High from 2, Medium from 1, otherwise Low. ```text Baseline (sorted): [8, 10, 11, 12, 13, 14, 16, 20] Q1 11, Q3 16, IQR 5 Bounds: 11 − 7.5 = 3.5 to 16 + 7.5 = 23.5 Run: 35 rows → above 23.5 → anomaly ``` Suits history with occasional spikes. ### Percentage change Beacon compares the run with the **mean of the baseline** (not with the previous run): `change = |x − mean| / mean`. The run is an anomaly when the change is above the sensitivity's threshold and its direction has alerting turned on. | Sensitivity | Threshold | |-------------|-----------| | High | 15% | | Medium | 25% | | Low | 40% | Severity by change: Critical from 100%, High from 50%, Medium from 25%, otherwise Low. When the baseline mean is 0, any run with rows is an anomaly (severity High). ```text Baseline mean: 100 Run: 140 rows → change 40% Medium: 40% > 25% → anomaly, severity Medium ``` Suits business volumes where a relative change is what matters. ### Choosing a method | Data | Method | |------|--------| | Steady counts with little variation | Standard deviation | | Counts with occasional extreme values | IQR | | Volumes where a relative change matters (orders, sign-ups) | Percentage change | ## Tuning | Problem | What to change | |---------|----------------| | Alerts on normal variation | Lower the sensitivity (Low), try IQR, or lengthen the lookback. | | Real changes go unnoticed | Raise the sensitivity (High), or use Percentage change for sudden jumps. | | Detection never starts | The subscription has fewer baseline runs in the lookback window than **Minimum data points**. Wait for more runs, or create the subscription again with a lower minimum or a longer lookback. | | Only increases (or only decreases) matter | Turn off the other direction. | The lookback window counts days, not runs. A subscription that runs hourly with a 7-day lookback has up to 168 baseline values; one that runs daily has 7. Set **Minimum data points** to what the schedule can deliver within the lookback. ## Where you see it - **Subscription page, Anomaly detection tab** (shown while detection is on). The method, sensitivity, lookback, minimum data points and alert directions; a chart of the last 30 days of row counts with the baseline and the lower and upper thresholds; and the recent detection events. - **Control Tower.** An **Anomaly** badge on subscriptions with detection on, a per-day sparkline of anomaly events and the anomaly count, and the recent anomalies in the row detail. See [Control Tower](https://moberghr.github.io/beacon/features/control-tower/). The chart route is `GET /beacon/api/subscriptions/{id}/anomaly-chart?days=30`. It needs read access to the subscription, and its run points cover only runs the caller may read. ## Data model Three entities in the `beacon` schema hold anomaly data (PostgreSQL uses snake_case names): | Entity | Holds | |--------|-------| | `AnomalyConfig` | One per subscription: `Enabled`, `DetectionMethod`, `Sensitivity`, `LookbackDays`, `AlertOnIncrease`, `AlertOnDecrease`, `MinimumDataPoints`. | | `AnomalyBaseline` | One per stored run: `SubscriptionId`, `ExecutionTime`, `MetricValue` (the row count). | | `AnomalyEvent` | One per detected anomaly: `SubscriptionId`, `NotificationId`, `DetectedTime`, `CurrentValue`, `BaselineMean`, `BaselineStdDev`, `ZScore`, `Severity`, `Explanation`, `Acknowledged`, `AcknowledgedBy`. | ## Service API `IAnomalyDetectionService` (in `Beacon.Core.Services`) is what the subscription run uses. A host can call it in-process: | Method | What it does | |--------|--------------| | `EvaluateAnomalyAsync(subscriptionId, rowCount)` | Evaluates a row count against the stored baseline. | | `StoreBaselineAsync(subscriptionId, metricValue, executionTime)` | Stores a baseline value. | | `GetHistoricalMetricsAsync(subscriptionId, lookbackDays)` | Baseline values of the window, oldest first. | | `RecordAnomalyEventAsync(subscriptionId, evaluation, notificationId)` | Records an anomaly event. | | `GetAnomalyConfigAsync(subscriptionId)` | The configuration as `AnomalyConfigData`, or `null`. | | `SaveAnomalyConfigAsync(AnomalyConfigData)` | Creates or updates the configuration of a subscription. | | `GetAnomalyChartDataAsync(subscriptionId, lookbackDays, runScope)` | Chart data, limited to the runs `runScope` may read. | `AnomalyEvaluationResult` has `IsAnomaly`, `CurrentValue`, `BaselineMean`, `BaselineStdDev`, `ZScore`, `Explanation`, `Severity` and `HistoricalDataPoints`. `BaselineStdDev` and `ZScore` are set by the standard deviation method only. ```csharp await anomalyDetectionService.SaveAnomalyConfigAsync(new AnomalyConfigData { SubscriptionId = subscriptionId, Enabled = true, DetectionMethod = AnomalyDetectionMethod.PercentageChange, Sensitivity = AnomalySensitivity.Medium, LookbackDays = 30, MinimumDataPoints = 14, AlertOnIncrease = true, AlertOnDecrease = true }, cancellationToken); ``` `SaveAnomalyConfigAsync` does not check who calls it; a host that exposes it decides who may change a subscription's configuration. ## Related - [Subscriptions](https://moberghr.github.io/beacon/features/subscriptions/): where anomaly detection is configured - [Control Tower](https://moberghr.github.io/beacon/features/control-tower/): anomaly counts and sparklines across subscriptions - [Notifications](https://moberghr.github.io/beacon/features/notifications/): what recipients receive - [Tasks](https://moberghr.github.io/beacon/features/tasks/): work items opened by runs with rows --- # Data Quality > Data contracts with scheduled, rule-based checks that produce a weighted quality score, track trends over time, and alert on threshold breaches. Source: https://moberghr.github.io/beacon/features/data-quality/ Beacon monitors the health of your tables through **data contracts**: scheduled, rule-based checks that produce a weighted quality score, track trends over time, and alert your team when a table drops below its agreed threshold. ## Overview A **data contract** is an agreement about the expected state of one table: how fresh its data should be, how many rows it should hold, which columns must be unique or non-null, and so on. Each contract: - targets **one table** (schema and table) on one database [data source](https://moberghr.github.io/beacon/features/data-sources/); - contains one or more **rules**, each with its own severity and weight; - runs on a **cron schedule** and on demand; - produces an **overall score (0–100%)** per evaluation, a severity- and weight-adjusted average of the rule scores; - optionally **alerts recipients** when the score falls below a failure threshold. Every evaluation is stored, so each contract has a full history, and each table has a latest score with a trend (improving, stable, degrading). :::note Data contracts evaluate on **PostgreSQL, SQL Server and MySQL** data sources. On other database engines their rules fail with "Rule type … is not supported on …". A contract on a non-database data source (a REST API, for example) fails with "Data contract's data source must be a database type". ::: ## Ownership and access A contract's **owner** is the user who created it (an API key's owner, for a key), or the user it was handed to. Only the owner or an Admin can change, disable, retarget, evaluate on demand (**Evaluate now**) or delete it. To anyone but an Admin, a contract that does not exist is refused (`403`) like one that is not theirs. An Admin or a UserAdmin gives a contract a new owner with **Change owner** on its page (`PUT /beacon/api/data-quality/contracts/{id}/owner` with `{ "userId": }`). The new owner must be an existing, enabled user who may run saved SQL in a project that links the contract's data source; otherwise the request is refused (`409`) with a message saying so, because every scheduled evaluation would be skipped. If someone else changed the owner, or archived the contract, since it was read, nothing is changed and the call answers `409`; choosing the owner the contract already has changes nothing. Contract lists and details name the owner by display name, or as "User {id}" (`hasOwner`, `ownerDisplayName`), never by user name or external id. A contract belongs to the projects that link its data source. It is listed, evaluated and changed only through such a project the caller may use (see [Security model](https://moberghr.github.io/beacon/security/)); in a restricted project an Editor deletes only contracts they own. While Beacon user management is on, a caller without a Beacon user (a host principal, or a bearer token not bound to a user) cannot create a contract (`403`). ### Scheduled evaluations run as the owner Right before any rule runs, Beacon checks that the owner, as they are now, may still run saved SQL in a project that links the contract's data source. When the owner may not (disabled or archived, no longer a member of a restricted project, or without the role), the evaluation is skipped: - no rule runs, and no score, history or alert is written; - the job does not fail, and a Warning is logged (event 7111 `ScheduledWorkSkipped`, ids only); - the contract records the skip until an evaluation runs again (`ownerNotAuthorized`): its row on the Data Quality page shows **Owner not authorized**, its page shows a warning with **Change owner**, the overview counts such contracts (**Skipped (owner)**, `ownerNotAuthorizedContracts`), and the [Control Tower](https://moberghr.github.io/beacon/features/control-tower/) says how many contracts are not evaluated. A contract without an owner is evaluated only while its data source is openly reachable, and never with `Beacon:Authorization:RequireProjectMembership: true`. **Evaluate now** runs as you, the caller. ## Rule types Each rule has a **type**, an optional **column** and a **configuration** (JSON). | Rule type | What it checks | Example configuration | |-----------|----------------|-----------------------| | **Freshness** | The newest value in a timestamp column is no older than a maximum age | `{"column": "updated_at", "maxAgeMinutes": 60}` | | **Volume** | Row count is within an expected range (`minRows`, `maxRows`, or both) | `{"minRows": 100}` | | **Null rate** | Percentage of NULLs in a column stays at or below a threshold | `{"column": "email", "maxNullPercent": 5}` | | **Uniqueness** | A column contains no duplicate values | `{"column": "id"}` | | **Referential** | Every non-null value in a column exists in a reference table (no orphans) | `{"column": "user_id", "referenceTable": "users", "referenceColumn": "id"}` | | **Range** | Numeric values stay between `min` and `max` (either bound optional) | `{"column": "age", "min": "0", "max": "150"}` | | **Pattern** | All values match a pattern | `{"column": "email", "pattern": "^[^@]+@[^@]+$"}` | | **Custom SQL** | Any check you can express as SQL | `{"sql": "SELECT CASE WHEN COUNT(*) > 0 THEN 1 ELSE 0 END AS passed FROM ..."}` | You do not need `schema` or `table` in the configuration: Beacon takes them from the contract's target table. **Referential** rules default the reference table to the contract's schema; add `"referenceSchema": "..."` for another schema. **Rules stay on the contract's data source.** The contract's table, a rule's column, and a Referential rule's reference table and column must be objects of the contract's data source as its stored metadata lists them. - A name that differs from the metadata only in case is resolved to the metadata's spelling, both when the contract is saved and when a rule stored earlier is evaluated. A name that matches several names differing only in case (different objects on a case-sensitive server) has to be written exactly. - Saving a contract that names anything else answers `400` ("Table … is not in the data source's metadata …", "Column … is not a column of …", "… matches several …"). Columns are checked where the metadata lists the table's columns. - The generated SQL quotes these names as identifiers of the data source, so a rule cannot reach another schema, database or catalog the data source's login can see. - The same check runs on every evaluation: a rule that names an object outside the metadata is recorded as failed and not run. Refresh the data source's metadata first when the table is new. :::note[Pattern matching depends on the engine] On PostgreSQL the pattern is a regular expression (`!~`), on MySQL it uses `NOT REGEXP`, and on SQL Server it is evaluated with `NOT LIKE`. Use `LIKE` wildcards (`%`, `_`) for SQL Server sources, not regex. ::: ### Custom SQL rules A Custom SQL rule runs your query and reads the first row of the result. - The query must be a single read-only `SELECT` in the data source's SQL dialect. It is checked when the contract is saved and again on every evaluation, and runs through the same read-only path as MCP queries (a `READ ONLY` transaction on PostgreSQL and MySQL). - Beacon caps the result at one row itself, so the query must not set its own outer `LIMIT`, `TOP` or `FETCH`. - It may name only the data source's own database: a name qualified with another database, catalog or project is refused on save, and the rule is recorded as failed on evaluation. - Only an **Admin** can create, change or delete a contract that contains a Custom SQL rule. - Custom SQL rules are not available on host-managed data sources. Return these columns: | Column | Required | Meaning | |--------|----------|---------| | `passed` | Yes | `1` = pass, `0` = fail | | `score` | No | Score 0–100; defaults to 100 on pass, 0 on fail | | `actual_value` | No | Shown as the observed value in results | | `message` | No | Shown as the result message | `actual_value` and `message` reach the evaluation history and the contract's alerts. A PII column read into them (by a PII name such as `email`, an alias over one such as `MAX(email) AS actual_value`, or a project's custom PII pattern) is masked before the result is stored, following the PII detection setting of the projects that link the data source (the strictest applies). `passed` and `score` are read as numbers. If you keep reusable checks as saved [queries](https://moberghr.github.io/beacon/features/queries/), a Custom SQL rule folds the same logic into a quality score. ### Approval of Custom SQL The [approval workflow](https://moberghr.github.io/beacon/security/approvals/) is off by default; a host turns it on in code with `options.EnableApprovalWorkflow(...)`. While it is on, its `CoversCustomSqlRules` switch (on by default) makes the SQL a Custom SQL rule runs a reviewed change: - **A Custom SQL rule that is new, or whose SQL or data source changes, waits for a second person's approval** and is not evaluated meanwhile. It is left out like a disabled rule (each evaluation logs a Warning with the contract and rule ids), and the contract's other rules keep running. The contract page marks a waiting rule **Awaiting approval**. - **An evaluation that leaves rules out is partial.** It records how many enabled rules it left out (`awaitingApprovalRules`, `isPartial`), and the contract page marks it **Partial** with that count. A partial score **does not replace the table's score**: the contract's latest score and the table's trend stay as they were, so a held rule never improves them silently. For a contract that alerts, a scheduled partial evaluation at or above the failure threshold logs a Warning; one below the threshold alerts as usual, and the alert's payload carries the number of rules left out. - **A contract whose every enabled rule waits is not evaluated.** **Evaluate now** answers `400` with *"Every enabled rule of this data contract is a Custom SQL rule waiting for approval, so it was not evaluated. Approve the change on the Approvals page, or enable a rule that does not wait."*; the scheduled run is skipped with a Warning. No history, score or alert is written. - **The waiting rules form one change request per contract**, submitted with the save (the Admin who saved it is the requester). Creating a contract with Custom SQL rules answers with `awaitingApproval: true` and the message *"Submitted for approval: the contract's Custom SQL rules are not evaluated until a second person approves them."* - **An Approver (or an Admin) other than the requester reviews it** on the **Approvals** page under **Custom SQL rules**, which shows the contract's Custom SQL rules before and after the change, and approves exactly that SQL on that data source; a rule changed again since is not approved (`409`). Someone without the Approver permission who may read the contract's data source can open the request, but cannot decide it. - **An unchanged rule keeps its approval.** A save rewrites the rules, but a rule whose SQL and data source stay the same keeps its state (approved, waiting or ungoverned), so changing a schedule or a threshold submits nothing. A rule whose SQL changes keeps the SQL approved for it (by its name), so putting that SQL back needs no approval. A rule renamed and changed in the same save is a new rule. Moving the contract to another data source makes every Custom SQL rule wait again. Removing the waiting rules, or putting their approved SQL back, withdraws the requester's own pending request; a change of the waiting rules while another person's request waits is refused (`409`). - **Concurrent saves.** A save that finds the contract gained a Custom SQL rule while it was being saved is refused (`409`, *"The contract's rules changed while it was being saved, and a change of its Custom SQL rules waits for approval: reload it and save again."*); nothing is written. - **A rejected change** leaves its rules out of evaluations until they are changed back to approved SQL, removed, or saved again (which submits them again). The contract page shows the rejection with the reviewer's comment (`lastApprovalDecision` while no request is pending). - **Alerts skip recipients that wait.** A recipient whose destination waits for approval (see [Notifications](https://moberghr.github.io/beacon/features/notifications/)) receives no alert. The evaluation records how many were skipped (`heldAlertRecipients`), and an alert that every recipient waits for is logged as a Warning instead of being sent. :::caution[Upgrading] Custom SQL rules that exist when the workflow is turned on are not reviewed: they run as before, as long as their SQL and data source stay the same. A rule saved while the workflow is off is not reviewed later. Turning `CoversCustomSqlRules` (or the workflow) off again lets waiting and rejected rules be evaluated as they are; startup logs their count. ::: ### Severity and weight | Setting | Values | Effect | |---------|--------|--------| | **Severity** | Low, Medium, High, Critical | Multiplies the rule's influence: Low ×1, Medium ×2, High ×3, Critical ×4 | | **Weight** | The dialog accepts 0.1–10 (default 1.0) | Fine-grained influence within the same severity | | **Enabled** | on or off per rule | Disabled rules are skipped and do not affect the score | ## How evaluations run 1. **Scheduled.** Every enabled contract gets a recurring job on the host's scheduler, driven by its cron expression. The job is created when you save an enabled contract, updated when the schedule changes, and removed when you disable or delete the contract. 2. **On demand.** Click **Evaluate now** on the contract page (or `POST /beacon/api/data-quality/contracts/{id}/evaluate`). During an evaluation, Beacon: 1. loads the contract and its **enabled** rules; 2. generates engine-specific SQL for each rule and runs it against the data source (60-second timeout per rule); 3. turns each result into pass or fail, a rule score, an actual and expected value, and a message; 4. computes the overall score, stores the evaluation with all rule results, and updates the table's latest score and trend; 5. sends alerts if the score is below the failure threshold (scheduled runs only; see [Alerting](#alerting)). If one rule fails (bad configuration, missing column, timeout), it is recorded as **failed with score 0** and the other rules still run. A contract with no enabled rules is not evaluated: it keeps its last score, records no history and sends no alert. **Evaluate now** reports that it has no enabled rules, and the scheduled run is skipped. ## Scoring ### Per-rule score Most rule types score **100 on pass and 0 on fail**. Three score differently: | Rule type | Score on failure | |-----------|------------------| | **Null rate** | `max(0, 100 − (actualNullPercent − maxNullPercent))`: small overruns cost little | | **Range** | `(1 − outOfRangeRows / totalRows) × 100`: proportional to the rows that violate the range | | **Custom SQL** | What your query returns in `score` (default 100 or 0) | ### Overall score The overall score is a weighted average of rule scores. Each rule's effective weight is: ```text effectiveWeight = ruleWeight × severityMultiplier ``` with Critical = 4, High = 3, Medium = 2, Low = 1. Then: ```text overallScore = Σ(ruleScore × effectiveWeight) / Σ(effectiveWeight) ``` rounded to two decimals. Custom SQL rules waiting for approval are not part of the score. **Example:** a passing Critical freshness rule (weight 1, score 100) and a failing Low volume rule (weight 1, score 0) score `(100×4 + 0×1) / (4+1) = 80%`. ### Table scores and trends After every evaluation that is not partial, the target table's **latest score** is updated (one score per data source, schema and table). Beacon keeps the previous score and derives a trend: | Trend | Condition | |-------|-----------| | **Improving** | New score more than 1 point above the previous | | **Stable** | Within ±1 point | | **Degrading** | New score more than 1 point below the previous | Table scores also appear in the MCP `get_documentation` output for the table, with the rule results of the table's enabled contracts from their latest evaluation, each as a status Beacon writes itself (see [MCP server](https://moberghr.github.io/beacon/mcp/server/#get_documentation)). ## Alerting | Setting | Default | What it does | |---------|---------|--------------| | **Alert on failure** | on | Turns alerts for the contract on or off | | **Failure threshold (%)** | 80 | Alerts fire when the overall score drops **below** this value | | **Notification recipients** | none | Who is alerted (the same recipients [subscriptions](https://moberghr.github.io/beacon/features/subscriptions/) use) | When a **scheduled** evaluation scores below the threshold, Beacon sends a notification titled `[Data Quality] {contract name}` to every recipient. The payload has the overall score, the threshold, the target table, pass and fail counts, and every failed rule (name, score, expected and actual value, message). For channels, see [Notifications](https://moberghr.github.io/beacon/features/notifications/). :::note **Evaluate now** records the evaluation and updates the score but sends no alert. Only scheduled runs alert. ::: ## Using the UI ### Data Quality page Open **Data Quality** in the navigation (`/beacon/data-quality`). The page shows: - **overview cards per data source**: average score, table count, **Healthy** tables (score of at least 80), **Degrading** tables, **Contracts**, **Skipped (owner)** when any, and per-table scores; - **the contracts list**, paged on the server, newest first: each contract with its target table, latest score and state, and **View**, **Edit** and **Delete**. ### Create a contract 1. Click **New contract**. 2. Fill in the details: | Field | Required | Example | |-------|----------|---------| | **Contract name** | Yes | `Orders freshness & volume` | | **Description** | No | | | **Data source** | Yes | A PostgreSQL, SQL Server or MySQL data source | | **Schema name** / **Table name** | Yes | `public` / `orders` | | **Cron expression** | Yes | `0 */6 * * *` (every 6 hours) | | **Failure threshold (%)** | No | `80` | | **Enabled** | | on: the schedule is active | | **Alert on failure** | | on: alert below the threshold | | **Notification recipients** | | Shown while **Alert on failure** is on | 3. Add one or more rules: **Name**, **Type**, **Column (optional)**, **Severity**, **Weight** and **Configuration (JSON)** (the dialog shows an example for the type). 4. Click **Create**. An enabled contract is scheduled at once. ### Contract page The contract page (`/beacon/data-quality/{id}`) shows the KPIs **Latest score**, **Rules**, **Schedule**, **Failure threshold** and **Owner**, and three tabs: - **Rules**: the rule set; - **Evaluations**: history, newest first, with overall score, pass and fail counts and execution time, a **Partial** marker with the number of Custom SQL rules left out, and the number of alert recipients skipped; - **Latest results**: per-rule results of the most recent evaluation (pass or fail, score, expected and actual value, message). **Evaluate now**, **Edit**, **Change owner** (Admins and UserAdmins) and **Delete** sit in the header. Deleting a contract archives it and removes its scheduled job. :::note **Editing a contract replaces its rule set.** A rule whose definition is unchanged stays as it is; the changed and removed rules are retired, not deleted: they are never evaluated again and no longer show on the contract, but they stay stored with their definition (a Custom SQL rule's SQL included), and the per-rule results of past evaluations stay in the **Evaluations** history under the name of the rule that produced them. Moving the contract to another data source retires every rule and removes their stored results, so the history never shows one source's results under another. Beacon purges nothing else, so a retired rule is kept as long as the evaluation results that point at it. A save that finds its rules already replaced by another save is refused with a conflict (409): reload the contract and save again. ::: ## REST API Routes under `/beacon/api/data-quality`. A scoped API key needs the Execute scope for `POST`, `PUT` and `DELETE`. | Method | Route | What it does | |--------|-------|--------------| | `GET` | `/overview` | Quality overview per data source (optional `dataSourceId`) | | `GET` | `/contracts` | List contracts, paged: `page` (zero-based), `pageSize` (default 20, at most 200), `sort` (default `-createdTime`), optional `dataSourceId` | | `GET` | `/contracts/{id}` | Contract detail with rules and latest score | | `POST` | `/contracts` | Create a contract | | `PUT` | `/contracts/{id}` | Update a contract (replaces rules and recipients) | | `PUT` | `/contracts/{id}/owner` | Give the contract a new owner (UserAdmin or Admin), body `{ "userId": … }` | | `DELETE` | `/contracts/{id}` | Archive a contract and remove its schedule | | `GET` | `/contracts/{id}/evaluations` | Evaluation history (optional `take`, default 20) | | `POST` | `/contracts/{id}/evaluate` | Run an evaluation now | Lists and the overview cover only contracts in projects you can read. ## Troubleshooting **A rule always fails with "Execution failed on the data source."** The rule's SQL could not run. The server's error text is not shown or stored, because it can quote row values. Check that the column exists and the data source connection works. Other failures name their cause: - "Invalid rule configuration: …": the configuration JSON is incomplete for the rule type. - "Rule type … is not supported on …": the rule type has no SQL for the data source's engine. - "Rule SQL was rejected: …": the SQL is not a single read-only `SELECT`, or reads something a host-managed source does not expose. - "Table … is not in the data source's metadata …" / "Column … is not a column of …": the contract or rule names an object the metadata does not list. Refresh the metadata or fix the name. "… matches several tables (columns) … that differ only in case": write the name exactly as the metadata spells it. - "Custom SQL must not set its own LIMIT, TOP or FETCH; Beacon reads one row.": remove the outer row limit. - "Rule timed out.": the query ran longer than 60 seconds. - "Rule result did not have the expected columns or types.": a Custom SQL rule did not return `passed` (and the optional columns) as described above. - "Query failed on the host database.": any failure on a host-managed data source. **"Data contract's data source must be a database type"** The contract targets a non-database data source (a REST API source, for example). **"This data contract has no enabled rules, so it was not evaluated."** Turn at least one rule's **Enabled** toggle on and save the contract. **No alerts despite a failing score** Alerts need **Alert on failure** on, at least one recipient, and an overall score strictly **below** the threshold. Only scheduled runs alert; **Evaluate now** does not. A recipient whose destination waits for approval receives nothing. **Pattern rule behaves differently across engines** PostgreSQL and MySQL treat the pattern as a regular expression; SQL Server uses `LIKE` syntax. **History shows fewer entries than expected** The history route returns the 20 most recent evaluations by default; pass a larger `take`. Rule-level details of old rules are removed when a contract is edited. --- # Data Migration > Copy rows from a read source into a database marked as a migration write target, in Insert, Upsert or Truncate mode. Source: https://moberghr.github.io/beacon/features/data-migration/ A migration job reads rows from a source through the same read-only path every Beacon query uses, then writes them into a table of a destination database. It is the only Beacon feature that writes to a connected database, so it is off unless the host turns it on, and it writes only into a data source an Admin marked for it. What a job does: - reads one source query (plain SQL, or a JSON list of steps across data sources) within its own row budget; - writes the rows into one destination table in one of three modes: **Insert only**, **Insert or update** (Upsert) or **Truncate & insert**; - records every run in its history: rows read, written and failed, duration and error. Typical uses: copy reference data into an analytics database, keep a reporting table refreshed, move rows between PostgreSQL, MySQL and SQL Server. ## Turning data migration on Data migration is off by default. The host turns it on in configuration: ```json { "Beacon": { "Migrations": { "Enabled": true, "MaxRows": 100000 } } } ``` | Key | Default | What it does | |-----|---------|--------------| | `Beacon:Migrations:Enabled` | `false` | Deployment lock. While `false`, nobody can create or run a job, an Admin included. | | `Beacon:Migrations:MaxRows` | `100000` | Row budget of a migration's source read (1 to 10,000,000), used in place of `Beacon:Query:MaxExecutionRows`. | While `Enabled` is `false`: - the `/beacon/api/migrations/*` routes are not mapped (every one answers `404`), and the UI shows no **Data Migration** entry, page or Home card; - the migration handlers refuse every request (`409`), also for an in-process caller that does not use the routes; - no data source can be marked as a migration write target (`409`). An existing target can still be returned to a read source. See [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/#data-migration). ## Migration write targets A job writes only into a data source an Admin marked as a **migration write target**: on the data source's page, choose **Mark as migration write target**, or call `PUT /beacon/api/data-sources/{id}/migration-writes` (Admin only). - Only a PostgreSQL, MySQL or SQL Server database can be a write target. A host-managed data source or one marked read-only cannot. - The host registers the target engine's connector, which brings its writer: `AddPostgreSqlConnector()` (`Moberg.Beacon.Connector.PostgreSql`), `AddSqlServerConnector()` (`Moberg.Beacon.Connector.SqlServer`) or `AddMySqlConnector()` (`Moberg.Beacon.Connector.MySql`). Without it a run fails, naming the package, before the target's credentials are decrypted. - Beacon never reads a write target. Query steps, previews, subscriptions, data quality, MCP tools, AI grounding, documentation and metadata scans refuse it or leave it out. Only the migration writer connects with its credentials, so only its login holds [write grants](#destination-privileges); every data source Beacon reads keeps a read-only login. - A job's declared source and the data source of every step it runs may not be a write target. To copy within one database, register it twice: once with a read-only login (the source) and once with the write login (the target). - Marking a data source that live queries, subscriptions or data contracts read is refused (`409`, with the count of each) unless the Admin confirms it (`"force": true`; the page asks a second time). Those readers fail once Beacon stops reading the source. - Returning a target to a read source asks for confirmation: Beacon reads it again with its stored login, so change that login to a read-only one first. - Every change is recorded in the settings history (`DataSource.{id}.AllowMigrationWrites`, before and after, and who) and as the `MigrationWritesChanged` security log event (7115, ids only). The flag is checked when a job is created, before every run, and by the writer before it decrypts the destination's credentials. A run started from the page or the API whose destination is not (or no longer) a write target, or that reads one, is refused with the reason (`400`). A run refused inside the migration service itself (an in-process caller, or a flag changed between the two checks) is recorded in the job's history as failed, with the reason. See [Data Sources](https://moberghr.github.io/beacon/features/data-sources/) for the data source side. ## Who can use it Data migration needs the **MigrationOperator** permission (an Admin holds every permission; see [Security model](https://moberghr.github.io/beacon/security/)). Listing jobs, opening one, creating, running and deleting jobs and reading the run history all need it. The **Data Migration** entry is hidden from other users. A MigrationOperator gets no bypass of project access. A job reads its declared source and the data source of every step in its query. Creating it, every run of it and deleting it need: - **Query** on each of those data sources and **Write** on the destination, through the projects that link them (`404` for a data source the caller cannot read, `403` for one it reads but may not query or write); - a destination that is **no more widely reachable** than any source: every live project that links the destination must link each source too, and an openly reachable destination needs openly reachable sources (`403` otherwise). Copying data never makes it readable to more people. The job list, a job and the run history show a MigrationOperator only jobs whose source and destination they can read (another job is not found, `404`). A failed run's error text, which can quote any data source the job reads, is replaced by a note unless the caller reads every one of them. An Admin is not checked. Refusals are logged with ids only. The Home page's migration summary shows Admins the totals of every job, MigrationOperators the totals of the jobs their list shows, and zeros to everyone else. ## Create a job 1. Open **Data Migration** (`/beacon/migration-jobs`). 2. Click **New job**. 3. Fill in the form and click **Create migration job**. | Field | Required | Default | What it does | |-------|----------|---------|--------------| | **Job name** | Yes | | Up to 200 characters. | | **Description** | Yes | | Up to 1,000 characters. | | **Source data source** | Yes | | The data source the job reads. Cannot be a write target. | | **Source SQL** | Yes | | The source query. See [Source query](#source-query). | | **Destination data source** | Yes | | Lists migration write targets only. | | **Destination table** | Yes | | For example `public.target_table`. | | **Migration mode** | Yes | Insert only | **Insert only**, **Insert or update** or **Truncate & insert**. See [Modes](#modes). Any other value is refused. | | **Cron schedule** | No | empty | Empty, or a standard 5-field cron expression (anything else is refused). Stored with the job; Beacon does not run it. See [Schedules](#schedules). | | Max retries (API only) | No | 3 | Not on the form. The API accepts 0 or 3 and refuses anything else; a failed run is never retried automatically. | | **Timeout (minutes)** | No | 30 | 1 to 1,440 (anything else is refused). A run that lasts longer is stopped and recorded as failed. | | **Enable migration job** | No | on | A disabled job does not run. | | Validate before execution (API only) | No | on | Not on the form. Every run checks its source SQL and destination table before it writes; the API refuses `false`. | | **Transformation script (optional)** | No | empty | Leave it empty. A job with a script fails when it runs. | The job is saved when the form is valid; its SQL is not run at creation. The destination table is checked for existence on every run. :::note A job cannot be edited after it is created; no route changes one. To change its query, destination, mode, schedule or whether it is enabled, create a new job and delete the old one. The old job's run history is kept. ::: ### Source query The source is read the way every query runs (see [Queries](https://moberghr.github.io/beacon/features/queries/)): through the read-only execution gate and the source's read-only path, within `Beacon:Query:StatementTimeoutSeconds` (default 120 seconds) per statement. The job's steps run as stored on the job, in the run itself: no query is saved to run them, so none is left behind. - **Row budget.** `Beacon:Migrations:MaxRows` replaces `Beacon:Query:MaxExecutionRows`. The cap is written into the source SQL and the read stops one row past it. A larger source fails the run, with a message naming `Beacon:Migrations:MaxRows`, before anything is written. Split the job (for example by date or key range) or raise the budget. A run holds every row it copies in memory, so size the budget to the host. - **Failed reads write nothing.** If any step or the final query fails or times out, the run is recorded as failed and the destination is not opened. A Truncate-mode job never empties its table on a failed read. - **Columns map by name.** The source's column names are used as the destination column names. - **Parameters.** A run from **Run now** or `POST /beacon/api/migrations/jobs/{id}/run` passes no parameter values. **Several data sources in one job.** The job's query text can also be a JSON array of steps, each naming its own data source. The form takes a single SQL statement; send the JSON form through `POST /beacon/api/migrations/jobs`: ```json [ { "Name": "Users", "DataSourceId": 3, "SqlValue": "SELECT user_id, email FROM users WHERE active = true" }, { "Name": "Orders", "DataSourceId": 7, "SqlValue": "SELECT user_id, COUNT(*) AS order_count FROM orders GROUP BY user_id" } ] ``` Each step is a read on its own data source, and the caller needs **Query** on each. Results of several data sources are joined in an in-memory SQLite database, as for any multi-step query. Anything that is not a JSON array is read as one step of plain SQL. ### Modes | Mode (UI label) | What it does | Needs | |-----------------|--------------|-------| | **Insert only** | Inserts every source row. A row that violates a key or constraint fails the write. | | | **Insert or update** (Upsert) | Inserts new rows and updates existing ones by primary key. | A primary key on the destination table, and every key column in each source row (otherwise the run fails). Without a primary key the run fails. | | **Truncate & insert** | Empties the destination table, then inserts every source row. | | :::caution **Truncate & insert** deletes every row in the destination table on each run. Run the job by hand against a test table before you enable it on real data. ::: How the write is done per engine: | Engine | Insert | Upsert | |--------|--------|--------| | PostgreSQL | `COPY … FROM STDIN (FORMAT BINARY)` | `COPY` into a temporary table, then `INSERT … ON CONFLICT` | | SQL Server | bulk copy, keeping identity values | bulk copy into a session temporary table, then `MERGE` (a matched row is updated only when a value changed) | | MySQL | multi-row parameterised `INSERT … VALUES (…), (…)` | the same, with `ON DUPLICATE KEY UPDATE col = VALUES(col)` | A run writes in one transaction on every engine: a failed write fails the run and rolls every row back, so a run either completes or writes nothing. On MySQL this holds for a transactional table (InnoDB); a MyISAM or MEMORY table cannot roll back, so a failed run keeps the rows written before the failure. Truncate & insert empties a MySQL table with `TRUNCATE TABLE`. The server refuses it for a table that another table references by foreign key, whatever the write login can see of that table, so the run fails and nothing is emptied or cascaded; a table's foreign key to itself does not stop it. On MySQL `TRUNCATE` commits implicitly and cannot be rolled back, so Beacon starts the transaction again right after it: a Truncate & insert run that fails after the table was emptied leaves the table empty, never partly written. Run it again once the cause is fixed. Because the table is not locked between the clear and the commit, two Truncate & insert runs into the same MySQL table at the same time can both write their rows; schedule jobs that write the same table so they do not overlap. The rows are written as the source returns them: - The columns written are those of all source rows. A NULL, or a column a row does not carry, writes NULL, never the column's default; in Upsert mode that NULL replaces the existing value. Rows from one query all carry the same columns. - SQL Server keeps the source's identity values (the table does not generate new ones) and fires the table's insert triggers, and checks `CHECK` and foreign-key constraints: a violating row fails the run, which rolls back. - In Upsert mode two source rows with the same primary key in one batch of 10,000 fail the run on PostgreSQL and SQL Server, which rolls back; on MySQL the later row wins. - A SQL Server table named without a schema is the `dbo` one. - A PostgreSQL table named without a schema is the one the write target's connection finds on its `search_path`: the first schema of `current_schemas(false)` that has a table of that name. The run checks, reads the key of and writes that table; a same-named table in a schema off the path is not used, and a name found on no schema of the path fails the run as a missing table. - MySQL sends at most 1,000 rows, 60,000 values or about 1 MB of values per statement, under MySQL 5.7's default 4 MB `max_allowed_packet`. Its Upsert matches on any unique key of the table, not only the primary key (how `ON DUPLICATE KEY UPDATE` works), and keeps `VALUES(col)`, which MariaDB and MySQL 5.7 understand; MySQL 8.0.20 and later accept it with a deprecation warning. ### Schedules The cron schedule is stored with the job and shown on it. Beacon does not start migration runs from it by itself: a run starts from **Run now**, `POST /beacon/api/migrations/jobs/{id}/runs` or `POST /beacon/api/migrations/jobs/{id}/run`. A host that wants scheduled runs starts them from its own background job (see [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/)); the approval and write-target checks, the timeout and the one-run-at-a-time rule apply to those runs too. `IMigrationService.ExecuteMigrationJob` answers a run that overlaps one starting or in progress with a failed response whose `RunInProgress` is `true` (execution id 0, nothing run or recorded in the history); it does not throw. ## Run a job 1. Open the job from the jobs list. 2. Click **Run now**. **Run now** is disabled while the job is disabled or awaiting approval. The run is a background task: the button shows it queued and running, and when it ends the page reads the job's runs again and says how it went (the run history has its counts and error). - **As a background task** (what the page's button does): `POST /beacon/api/migrations/jobs/{id}/runs` answers `202` at once with `{ taskId, jobId, state }` and a `Location` header, after the same checks as a run in the request (the deployment lock, project access, write targets, an existing enabled job). Read its progress with `GET /beacon/api/migrations/jobs/{id}/runs/{taskId}` (MigrationOperator, with the run's access checks; `state`: `0` queued, `1` running, `2` succeeded, `3` failed, `4` skipped; `warningCount` is the rows that failed in a run that finished; `failureCode` is `run_failed` for a run that failed, whose history row says why). The job runs as the user who started it and checks again, right before the run, that they still hold MigrationOperator and meet the job's access rules: if not, it is skipped (`forbidden`, on record as a `refused` `migrations.job_run_ended`) and nothing runs. A MigrationOperator role stored in Beacon is checked live. A permission only an identity-provider app role grants (`Beacon:Authorization:IdpRoles`) lives on the sign-in, which the job does not have: the start records that it came from there (`migrations.job_run_requested` carries `permissionSource`), and the job relies on it only within 15 minutes of the start (or the claim lifetime, if shorter), while the user is still enabled and its security stamp unchanged (a role change, a disable or "sign out everywhere" changes it); a run task that waits longer is skipped (`forbidden`). A job has one run task queued or running at a time (a second start answers `409`); a run task that finds a run of the job already in progress (started another way) is skipped (`busy`). The run itself is the same as in the request: the one-run-at-a-time claim, the destination lock, the job's timeout, the commit that is never cancelled and the history row that is always closed all apply. A run task left running by a host that stopped is given up (`abandoned`) when the next one starts or its progress is read, once the job's run history shows its run ended (closed, or left running past the job's timeout plus 10 minutes) or it did not reach its run, or no run of it began, within five minutes; runs started another way before the task reached its own are not counted. At the latest it is given up after the longest timeout (1,440 minutes) plus 10 minutes. A task given up before its run does not start one, and a task that still ends records its own outcome over `abandoned`. The job runs through your host's [`IBeaconScheduler.EnqueueBackgroundTask`](https://moberghr.github.io/beacon/integrate/background-jobs/); a host that does not implement it runs the migration in the request, and the `202` then carries the state it finished in. - **In the request** (kept for API callers of Beacon 5.0): `POST /beacon/api/migrations/jobs/{id}/run` answers with the run's counts once it has ended, which holds the request open for as long as the run takes. - **One run at a time.** A job runs once at a time, across every host: a run of a job that already has a run starting or in progress is refused with `409 Conflict` before it reads or writes anything, and adds no run to the history (the refusal is a security event with reason `run_in_progress`). - **Destination lock.** Inside its write transaction, before it checks or writes the table, a run takes a lock on its job in the destination database, without waiting: a PostgreSQL transaction-level advisory lock (`pg_try_advisory_xact_lock`), a SQL Server application lock owned by the transaction (`sp_getapplock`) or a MySQL named lock (`GET_LOCK`, released after the commit or rollback). If another run of the job holds it (a run whose host paused past the in-progress window while a new run started), the run fails with *"Nothing was written: another run of this job holds its destination's migration lock"* (security event reason `destination_busy`) and writes nothing. None of these locks needs a grant beyond the login's. - **A commit that cannot be confirmed.** If the destination's commit fails (the connection is lost during it, say), Beacon cannot tell whether the rows were written. The run is recorded as failed with *"The destination's commit could not be confirmed: check the destination table before running this job again"* (reason `commit_unknown`); nothing is rolled back or retried. Look at the table before you run the job again, especially in Insert mode. - **A host that stops during a run.** Its run stays **Running** and blocks the job until the job's timeout and 10 more minutes have passed since it started. The next run after that closes it as **Failed** (*"Abandoned: the host stopped during the run"*, security event reason `abandoned`); nothing needs to be done by hand. - **Timeout.** A run is stopped after the job's timeout (1 to 1,440 minutes; a job stored with another value is held to that range) and recorded as failed with *"The run timed out after N minutes"*. The write is one transaction, so a run stopped before it commits writes nothing (except a MySQL Truncate & insert, see [Modes](#modes)). ## Jobs and run history The jobs list (`/beacon/migration-jobs`) is paged on the server and shows **Id**, **Name**, **Source → destination**, **Mode**, **Enabled**, **Schedule** and **Created**, with a delete action per row. **History** opens the run history of all jobs. A job's page shows its definition (source, destination, mode, schedule, enabled, created), **Status**, **Last run** and the successes and failures among its recent runs, and its runs with **Started**, **Status**, **Duration**, **Rows**, **Retry** and **Error**. The run history page (`/beacon/migration-history`) lists runs of every job you can read with **Started**, **Job**, **Status**, **Rows**, **Failed** and **Rows/s**. | Status | Meaning | |--------|---------| | Running | The run has started. | | Completed | Every row was written. | | Partial | Recorded by earlier versions, whose MySQL writes fell back to row by row; runs no longer end in it. The job page labels it **Partial success**. | | Failed | The read or the write failed, the run timed out or was abandoned by a host that stopped, or the run was refused (not approved, not a write target). The error says why. | ### Delete a job Click the delete action on the row, or **Delete** on the job's page, and confirm. The job is archived; its run history is kept. ## Approval of new jobs The [approval workflow](https://moberghr.github.io/beacon/security/approvals/) is off by default; a host turns it on in code with `options.EnableApprovalWorkflow(...)`. While it is on, its `CoversMigrationJobs` switch (on by default) makes every new job wait for a second person's approval before it writes anything: - **A new job is submitted for approval**, with its creator as the requester. The create answers with `awaitingApproval: true` and the message *"Submitted for approval: the job does not run, manually or on its schedule, until a second person approves it."* A caller Beacon cannot name (an API key without an active owner, for example) creates nothing (`403`) unless the host sets `AllowSelfApproval`. - **An Approver (or an Admin) other than the creator reviews it** on the **Approvals** page under **Migration jobs**. The review shows the definition: source and destination data sources, the source query (SQL or JSON steps), the destination table, the mode and the transformation script. The approval covers exactly that definition. Name, description, schedule, retry, timeout and enabled are not part of it. Someone without the Approver permission who reads both of the job's data sources can open the request to see what waits, but cannot decide it. - **No run starts before the approval.** A run is refused and recorded as failed with *"Not run: the job's definition waits for approval. A second person approves it on the approvals page; until then the job does not run."*, and a Warning is logged with the job id. The job shows **Awaiting approval** with a link to the request, and **Run now** is disabled. - **A job runs only the definition that was approved.** If the stored definition differs from the approved one (a change made outside Beacon), the job does not run until a matching definition is approved. - **A rejected job never runs.** Its page shows that the definition was rejected, with the reviewer's comment and the time of the decision (`lastApprovalDecision` on the job). Delete it and create the job again with a corrected definition, which is submitted again. - There is no withdraw action: an Approver rejects the request, or the creator deletes the job. - Nobody approves their own job unless the host sets `AllowSelfApproval`, and a request is decided once. :::caution[Upgrading] Jobs that exist when the workflow is turned on are not reviewed: they run as before. A job created while the workflow is off is not reviewed later. Turning `CoversMigrationJobs` (or the workflow) off again lets waiting and rejected jobs run as they are; startup logs their count. ::: ## Destination privileges Only a [migration write target](#migration-write-targets)'s login needs write grants. Grant it only what the job's mode uses, on the destination table only: | Mode | PostgreSQL | MySQL | SQL Server | |------|------------|-------|------------| | Insert only | `INSERT` (written with `COPY … FROM STDIN`) | `INSERT` | `INSERT` (bulk copy); `ALTER` on the table too when the job writes an identity column | | Insert or update | `INSERT`, `UPDATE`, `SELECT` on the written columns (`ON CONFLICT` reads them, also on a table whose columns are all key columns), and `TEMPORARY` on the database (rows are staged in a temporary table; PostgreSQL grants it to `PUBLIC` by default) | `INSERT`, `UPDATE`; `SELECT` on the first key column too when every column is a key column (the update assigns it to itself) | `SELECT`, `INSERT`, `UPDATE` (staged in a session temporary table); `ALTER` on the table too when the job writes an identity column (`IDENTITY_INSERT`) | | Truncate & insert | the `TRUNCATE` privilege, `INSERT` | `DROP` on the table (MySQL requires it to truncate), `INSERT` | `ALTER` on the table (SQL Server requires it to truncate), `INSERT` | On PostgreSQL every mode also needs `USAGE` on the table's schema. Every mode also reads `information_schema` to check that the table exists (and, for Upsert, its primary key). Some of these grants are DDL-grade (`ALTER`, `DROP`, the `TRUNCATE` privilege), which is why such a login belongs on a write target that Beacon never reads, not on an ordinary data source. ## REST API All routes are under `/beacon/api/migrations`, mapped only while `Beacon:Migrations:Enabled` is `true`, and need the MigrationOperator permission. | Method | Route | What it does | |--------|-------|--------------| | `GET` | `/jobs` | List jobs (paged: `page`, `pageSize`, `sort`). | | `GET` | `/jobs/{id}` | One job. | | `POST` | `/jobs` | Create a job. | | `POST` | `/jobs/{id}/runs` | Start a run of a job as a background task (`202` with its ticket). | | `GET` | `/jobs/{id}/runs/{taskId}` | The state of a run task. | | `POST` | `/jobs/{id}/run` | Run a job now, in the request (kept for 5.0 callers). | | `DELETE` | `/jobs/{id}` | Archive a job. | | `GET` | `/executions` | List runs (paged). | ## Troubleshooting | Symptom | Check | |---------|-------| | No **Data Migration** entry | `Beacon:Migrations:Enabled` is `true`, and you hold MigrationOperator (or are an Admin). | | The destination picker is empty | An Admin has marked the destination as a migration write target. | | **Run now** is disabled | The job is enabled and not awaiting approval. | | Run failed: another run holds the destination's migration lock | A run of the same job is still writing (its host was slow or paused). Wait for it to end, then run the job again. | | Run failed: the destination's commit could not be confirmed | Check the destination table for the run's rows before you run the job again. | | Run answers `409 Conflict` | Another run of the job is starting or in progress (or a host stopped during one less than the job's timeout plus 10 minutes ago). Wait for it to end, then run it again. | | Run failed on a job with a transformation script | Create the job again with **Transformation script** empty. | | Run failed: row budget exceeded | The source returned more than `Beacon:Migrations:MaxRows` rows. Split the job or raise the budget. | | Run failed: table does not exist | The destination table exists in the write target's database, and the name includes its schema if needed. | | Run failed: missing primary key columns | In Upsert mode, the source query returns every primary key column of the destination table. | | Run failed: permission denied on the destination | The write target's login has the [destination privileges](#destination-privileges) of the job's mode. | ## Related - [Data Sources](https://moberghr.github.io/beacon/features/data-sources/): connections and the write-target flag - [Queries](https://moberghr.github.io/beacon/features/queries/): how a source query is read - [Approvals](https://moberghr.github.io/beacon/security/approvals/): the change approval workflow - [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/#data-migration): `Beacon:Migrations` --- # AI Integration > How Beacon connects to an LLM provider (OpenAI, Anthropic, Azure OpenAI, AWS Bedrock), what it uses the model for, and what it sends. Source: https://moberghr.github.io/beacon/features/ai-integration/ :::caution[Experimental feature] Features that use a large language model can produce incorrect, incomplete or misleading results, including invalid SQL. Review generated documentation and SQL before you rely on it. ::: Beacon uses one configured LLM provider for every AI feature. This page covers what Beacon uses the provider for and what reaches it; full provider configuration for a host is in [Integrate → AI](https://moberghr.github.io/beacon/integrate/ai/). ## Turning AI on The AI layer lives in the `Beacon.AI` package. A host turns it on in two places: ```csharp var beacon = builder.Services.AddBeaconServices(builder.Configuration, options => { options.UseAI = true; // default false }); builder.Services.AddBeaconAI(builder.Configuration); ``` With `UseAI` off (the default), Beacon registers stand-ins for the AI-backed services it needs elsewhere, so the host starts and the non-AI features work; **Test connection** and the eval harness report that AI is unavailable. ## Providers | Provider | `Provider` value | Admin Settings card | What it needs | |---|---|---|---| | OpenAI | `OpenAI` | OpenAI | API key, model | | Anthropic | `Claude` | Anthropic | API key, model | | Azure OpenAI | `AzureOpenAI` | Azure OpenAI | Endpoint (your resource), API key, deployment name as the model | | AWS Bedrock | `Bedrock` | AWS Bedrock | Region, model id, AWS credentials; needs `Moberg.Beacon.AI.Bedrock` + `AddBeaconBedrock()` | The model is the string you enter: Beacon does not restrict or recommend model ids. OpenAI and Anthropic are called on their public API endpoints; there is no base-URL or proxy setting for them. Azure OpenAI uses your resource's endpoint, and Bedrock your AWS region, so you choose where requests go. For Bedrock, the authentication mode decides the credentials: the IAM role mode (the default) uses the host's default AWS credentials (an IAM role, environment variables), the access key mode an access key id and secret access key, and the temporary credentials mode those keys with a session token. An `ApiKey` that holds a pair as `accessKey:secretKey`, from an older install, is still used when the key fields are empty; in the IAM role mode a warning is logged when it is (see [Admin Settings](https://moberghr.github.io/beacon/security/admin-settings/)). ## Configuration ### Admin Settings An Admin configures the provider at runtime in **Admin Settings** (`/beacon/admin-settings`), section **AI provider**: pick a provider card, fill in the fields it shows, use **Test connection**, then **Save changes**. Only an Admin changes the connection (provider, endpoint, region, credentials) and the concurrency and per-minute limits; a SecurityAdmin can change the models and the other settings and test the stored connection. Secrets are stored encrypted. The REST routes are `GET /beacon/api/admin-settings` (Auditor or SecurityAdmin), `PUT /beacon/api/admin-settings` and `POST /beacon/api/admin-settings/test-llm` (SecurityAdmin). See [Admin Settings](https://moberghr.github.io/beacon/security/admin-settings/). A save swaps the provider in place: every caller injects `ILlmProvider`, registered as `DelegatingLlmProvider`, which forwards to the provider `LlmProviderManager` currently holds. When the new configuration cannot build a provider, the manager keeps the previous one and logs a warning. :::caution[After a restart] At startup the provider is built from `Beacon:LLM`; saving Admin Settings swaps it immediately. Keep `Beacon:LLM` in step with Admin Settings, or save them again after a restart. ::: ### Host configuration and limits The `Beacon:LLM` section (`Provider` = `OpenAI`, `Claude`, `AzureOpenAI` or `Bedrock`; `ApiKey`, `Endpoint`, `Region`, `SessionToken`, `Model`, `FastModel`, `Limits`) is described key by key in [Integrate → AI](https://moberghr.github.io/beacon/integrate/ai/). Never put a provider key in source or a committed `appsettings.json`. Every LLM call goes through `LlmRequestQueue`. It allows at most `Limits:MaxConcurrentRequests` calls at once (default 50; a value saved under **Max concurrent requests** in Admin Settings applies without a restart, and a lower one never cancels a running call); background work holds at most half of them. A call that finds no free slot within `Limits:InteractiveQueueWaitSeconds` (60; background work `Limits:BackgroundQueueWaitSeconds`, 300; at most an hour), or arrives while `Limits:MaxQueuedRequests` (200) interactive calls, or half as many background calls, already wait, fails with `LlmQueueSaturatedException` ("The LLM is busy. Try again later."). For Claude and Bedrock the queue is the only retry layer: a transient failure (rate limit, overload, unavailable, timeout, connection failure) is tried up to three times, waiting at least what `Retry-After` asks (up to 60 seconds), with a slot held only while an attempt runs; a call that still fails surfaces as `LlmTransientException` (an `AiServiceException`). OpenAI and Azure OpenAI retry themselves and the queue does not retry them again: a status they stopped retrying surfaces as an `HttpRequestException` carrying it, a connection failure or timeout as an `AiServiceException`. See [Integrate → AI](https://moberghr.github.io/beacon/integrate/ai/#the-request-queue). Tokens and requests per minute under **Rate limits & budget** on the Admin Settings page apply only while **Enforce requests/tokens per minute** is on (off by default; `0` is unlimited); the monthly budget is recorded, not enforced. The **Fast model** serves the short classification calls on the `ask` path (intent classification and data-source routing); when it is empty, equals the main model, cannot be built or fails when called, they use the main model. ## What Beacon uses the model for | Feature | Where it is described | |---|---| | The MCP `ask` tool and knowledge answers: routing, SQL generation and repair | [MCP Server](https://moberghr.github.io/beacon/mcp/server/#ask) | | The learning loop's lesson extraction, the optional eval judge and contextual-retrieval blurbs | [Knowledge Base & Grounding](https://moberghr.github.io/beacon/mcp/knowledge-base/) | | Project documentation | [below](#project-documentation) | | AI actors | [AI Actors](https://moberghr.github.io/beacon/features/ai-actors/) | Local embeddings for semantic search do not use the LLM provider: they run in-process from an ONNX model (see [Local embeddings](https://moberghr.github.io/beacon/mcp/knowledge-base/#local-embeddings)). ## Project documentation Beacon generates documentation per **project**, from the metadata of the data sources the project links and the code references found in its repositories. Generation runs as a background task that follows its requester (see below). The result has these sections: Project Overview, Business Domains, Data Model, Data Flows, Code Lineage, Data Quality, API Documentation (for API data sources) and Glossary. | Action | Route | Needs | |---|---|---| | Read the latest documentation and its history | `GET /beacon/api/projects/{id}/documentation` | Read in the project | | Generate or regenerate (background task, `202` with `{ taskId, jobId, state }`) | `POST /beacon/api/projects/{id}/documentation-generations` | Curate in the project | | Read a generation's progress (`state`: `0` queued, `1` running, `2` succeeded, `3` failed, `4` skipped; `failureCode` says why) | `GET /beacon/api/projects/{id}/documentation-generations/{taskId}` | Read in the project | | Generate or regenerate (5.0 route, kept; answers with the job id) | `POST /beacon/api/projects/{id}/generate-documentation` | Curate in the project | | Edit a section | `PUT /beacon/api/projects/documentation-sections/{id}` | Curate in the project | | Ask the LLM to rewrite a section (returns the new text, saves nothing) | `POST /beacon/api/projects/documentation-sections/{id}/instruct` | Query in the project | | Export as Markdown, or HTML with `asHtml=true` | `GET /beacon/api/projects/documentation/{id}/export` | Read in the project | Curate is held by project admin members and Admins (see [Who curates grounding](https://moberghr.github.io/beacon/mcp/knowledge-base/#who-curates-grounding)). In the UI, the **Documentation** tab of a project (`/beacon/projects/{id}`) shows the latest documentation, lets anyone who may read it download it as Markdown or HTML, and lets a curator generate it, edit its sections, or ask the LLM to change a section (the proposed text is shown and saved only after a confirmation). A generation runs as the user who started it: right before the first LLM call it checks again that they may still curate the project and query its data sources (a live Admin may), and is skipped (`forbidden`) if not. Only one generation of a project is queued or running at a time (`429` otherwise). A host whose scheduler implements `EnqueueBackgroundTask` runs it in its job runner; without one it runs in the request. **Curated sections.** A section a person changed, by editing it or by applying a documentation patch to it, records who changed it (`Users.Id`) and when (`editedTime` in the API) and is marked **Curated** in the UI. Regenerating keeps it: the new documentation takes the latest documentation's curated sections in place of what the run generated for their type, still marked curated, so regeneration never replaces curated text with fresh LLM output. A section that failed to generate holds a fixed "Generation failed for this section" text, never the provider's error. Each finished generation is recorded in the security audit as `curation.documentation_generated`. The documentation is also grounding: it is chunked, optionally embedded, and retrieved when an agent asks a question over MCP, so a better table description improves generated SQL (see [Documentation chunking](https://moberghr.github.io/beacon/mcp/knowledge-base/#documentation-chunking)). **Column values.** Generation lists distinct values of enum-like columns only while sample-value collection is on for the project and every project sharing the data source (off by default), never from PII or secret-like columns, and never under the deployment lock `Beacon:Mcp:ForbidRowDataToLlm`. Each prompt that carries such values is logged as event `7123`. ## What reaches the provider A prompt carries the question, schema (table and column names, types, descriptions, join paths) and your curated knowledge (documentation, glossary, verified examples, learned patterns). Row data (sample values, value matches, discovered column values, eval-judge rows, database errors) is off by default or screened, and every prompt that carries it is logged as event `7123` (`RowDataSentToLlm`, category `Beacon.AI.Services.LlmProviders.LlmRowDataEgress`) with identifiers only. The full table, and the lock `Beacon:Mcp:ForbidRowDataToLlm` that keeps all row data from the provider, are in [What reaches the LLM provider](https://moberghr.github.io/beacon/mcp/server/#what-reaches-the-llm-provider). Everything Beacon sends about a data source is read through the caller's authorized project: schema context, documentation, code references, glossary, golden examples, learned patterns and value matches. The AI services take that project as a parameter (`projectId` on `IKnowledgeGraphService` and `IValueGroundingService`), read a data source only through a project that links it, and never mix in another project's material from a shared data source. See [Project isolation](https://moberghr.github.io/beacon/mcp/knowledge-base/#project-isolation). ## Usage and cost records Beacon records token counts and estimated cost where a feature stores them: AI actor think cycles and plans (and the actor's totals). Documentation generation records no cost. Use your provider's own usage reporting for billing. ## Troubleshooting **"No active LLM provider" on Admin Settings.** No provider has been saved in Admin Settings yet. Pick a provider, test it and save. **AI calls fail after a restart although Admin Settings looks right.** The provider was built from `Beacon:LLM` at startup, which logs a warning when it cannot build one (for example Azure OpenAI without `Endpoint`, Bedrock without `Region`). Save Admin Settings again, and bring `Beacon:LLM` in step. **Test connection fails.** Check the key, the model or deployment name, and for Azure OpenAI the endpoint. For Bedrock, check the region, the model id and that the host's AWS credentials may invoke the model. **Test connection reports that AI is unavailable.** The host did not set `UseAI = true` or did not call `AddBeaconAI`. **Calls fail after retries.** Rate-limit and transient errors are tried up to four times on OpenAI and Azure OpenAI and three times on Claude and Bedrock; persistent failures usually mean the provider's own quota. Lower **Max concurrent requests** in Admin Settings, or turn on **Enforce requests/tokens per minute** with the provider's quota, or raise the quota with the provider. **Calls fail with "The LLM is busy. Try again later."** Every slot stayed taken for the whole wait limit, or too many calls were already waiting, or the enforced requests or tokens per minute left no room within the wait. Retry later, or raise **Max concurrent requests** or the per-minute limits (within the provider's quota), or the `Limits:*QueueWaitSeconds` / `Limits:MaxQueuedRequests` limits and restart. **Debug logging.** Raise the log level for the AI namespaces: ```json { "Logging": { "LogLevel": { "Beacon.AI.Services.LlmProviders": "Debug", "Beacon.AI.Services.Ai": "Debug", "Beacon.AI.Services.Documentation": "Debug" } } } ``` ## Related features - [Integrate → AI](https://moberghr.github.io/beacon/integrate/ai/): host configuration of the provider - [Admin Settings](https://moberghr.github.io/beacon/security/admin-settings/): the runtime provider configuration - [Knowledge Base & Grounding](https://moberghr.github.io/beacon/mcp/knowledge-base/): how documentation, join paths and glossary terms ground SQL generation - [MCP Server](https://moberghr.github.io/beacon/mcp/server/): the tools that use the provider - [AI Actors](https://moberghr.github.io/beacon/features/ai-actors/): LLM-driven monitoring agents - [Anomaly Detection](https://moberghr.github.io/beacon/features/anomaly-detection/): statistical detection, without an LLM --- # AI Actors > Experimental LLM-driven monitoring agents attached to a data source. They analyze it and propose queries and subscriptions; the proposals wait in a plan until a person approves them. Source: https://moberghr.github.io/beacon/features/ai-actors/ :::caution[Experimental feature] AI actors are experimental and driven by a large language model, which can produce incorrect or misleading analysis and invalid SQL. Review what an actor proposes before you act on it. ::: An **AI actor** is an LLM-driven monitoring agent attached to one data source. You give it a goal in plain English ("monitor transactions for unusual activity"); on each **think cycle** it reads the schema, its queries and recent results, writes an analysis, and chooses actions such as creating a query or a subscription. **Actions are proposed, not executed.** An actor has **Requires approval** on by default (the create request's `requiresApproval`; it cannot be changed afterwards). In that mode the initial setup, a think cycle and a refinement only **record** the actions the LLM chose: each is marked proposed on the cycle's execution record, the proposals are stored together as one [plan](#plans) awaiting approval, and nothing is created, refined or archived until a person approves that plan. An actor created with `requiresApproval: false` runs the actions it chooses as it chooses them; only an Admin can create one (anyone else gets `403`), and every creation is recorded in the security audit trail as `approvals.ai_actor_created` with the flag. AI actors need the AI layer and a configured provider (see [AI Integration](https://moberghr.github.io/beacon/features/ai-integration/)). ## Creating an actor Go to **AI Actors** (`/beacon/ai-actors`) and choose **New actor**: | Field | What it does | Default | |---|---|---| | **Name** | Display name | required | | **Data source** | The one data source the actor monitors | required | | **Instructions** | The monitoring goal | required | | **Additional context** | Business rules, table hints | none | | **Max queries (optional)** | Most queries the actor may create | 10 when left blank | | **Activate immediately** | Run the initial setup now | on | The REST endpoint is `POST /beacon/api/ai-actors`; it also takes `maxSubscriptionsPerQuery` (default 3), `defaultRecipientIds` and `requiresApproval` (default `true`; `false` takes an Admin; the response returns it), which the dialog does not show. With **Activate immediately** on, the actor becomes `Active` and runs its **initial setup**: the LLM reads the schema and your instructions and proposes a first set of queries and subscriptions. If setup fails, the actor moves to `Failed` with the error recorded. With it off, the actor stays in `Draft`. ### Creator and access The actor's **creator** is the signed-in caller (an API key's owner, for a key); a `createdByUserId` in the request is ignored. While Beacon user management is on, a caller without a Beacon user cannot create an actor (`403`). Only the creator or an Admin can pause, resume, archive, refine or run the actor, decide one of its plans, or run one of the subscriptions it manages on demand (`POST /beacon/api/subscriptions/{id}/execute`). An Admin or a UserAdmin can give an actor a new creator with **Change owner** on its page (`PUT /beacon/api/ai-actors/{id}/owner` with `{ "userId": }`). The new creator must be an existing, enabled user who may work ad hoc (Editor or above) in a project that links the actor's data source; otherwise the change answers `409`, because every background think cycle would be skipped. If someone else changed the creator, or archived the actor, since it was read, nothing is changed and the call answers `409`; choosing the creator the actor already has changes nothing. The actor's page names the creator by display name, or "User {id}" without one. An actor belongs to the projects that link its data source: the actor, its plans and its think cycles are visible only through such a project the caller may use, and refining it or running it on demand needs Query there (see [Project access](https://moberghr.github.io/beacon/security/authorization/#project-access)). To anyone but an Admin, an actor or plan that does not exist is refused (`403`) like one that is not theirs. An actor created without a recorded creator can be changed or run only by an Admin until an Admin sets its creator. ## Actor statuses | Status | Meaning | |---|---| | `Draft` | Created with **Activate immediately** off | | `Active` | Runs think cycles | | `Paused` | No background think cycles run | | `Failed` | Setup or a think cycle failed; the error is stored on the actor | | `Archived` | Soft-deleted | Pause applies to an `Active` actor; resume takes a `Paused` or `Failed` actor back to `Active`. On the actor's edit page (`/beacon/ai-actors/{id}/edit`) use **Pause actor** / **Resume actor**; the routes are `POST /beacon/api/ai-actors/{id}/pause` and `/resume`. Archive with `DELETE /beacon/api/ai-actors/{id}` (the **Archive** button on the actor's page, after a confirmation); the list shows archived actors with **Include archived**. ## The think cycle A think cycle is one reasoning pass, recorded as an execution: | Phase | What happens | |---|---| | `Analyzing` | Gathers the schema, the actor's queries and recent subscription results | | `Planning` | Sends that context to the LLM, which returns an analysis, findings and actions | | `Executing` | Records each chosen action as proposed (or runs it, for an actor that does not require approval) | | `Notifying` | Records notification intent when the LLM flagged important findings | | `Completed` | The cycle finished | | `Failed` | The cycle failed; the error is stored and the actor moves to `Failed` | | `Skipped` | Nothing ran: a background cycle whose creator may no longer work with the data source, or a cycle of an actor that requires approval while one of its plans is pending (the reason names the plan) | Phase checkpoints are saved before and after each LLM call, so a cycle that hangs or whose worker dies still shows where it stopped. The actions an actor can choose: | Action | Meaning | |---|---| | `CreateQuery` | A new monitoring query | | `CreateSubscription` | A subscription for one of its queries | | `RefineQuery` | New SQL for a query the actor owns | | `ArchiveQuery` | Retire one of its queries | | `ArchiveSubscription` | Retire one of its subscriptions | | `SendNotification` | Flag findings for notification | Each execution stores the decision summary, the analysis (markdown), key findings, each action with its reasoning, tokens used, estimated cost and the model. The actor's page (`/beacon/ai-actors/{id}`) shows think cycles, tokens used, total cost and the recent think cycles. **When cycles run.** - **On demand**: the actor's page has a **Think now** button, which starts the cycle as a background task: `POST /beacon/api/ai-actors/{id}/think-tasks` answers `202` with the task (its `jobId` correlates the `JobStatusChanged` events the user gets), and `GET .../think-tasks/{taskId}` reads its state (`Queued`, `Running`, `Succeeded`, `Failed` or `Skipped`, with a `failureCode`: `cooldown`, `plan_pending`, `inactive` (the actor was paused or archived meanwhile), `forbidden`, `not_found`, `abandoned`, `cancelled`, `failed`; never the provider's message, the cycle's own reason is on its execution). Only the creator or an Admin may start one; a second start while one is queued or running answers `429`. The task runs as you and re-checks that you may still run the actor and reach its data source right before the LLM call; if not it is skipped, never run as the system. The older `POST /beacon/api/ai-actors/{id}/think` runs the cycle inside the request and answers with its result (kept for 5.0 callers). Starting a task is recorded as `approvals.ai_actor_think_started`. - **After a subscription run**: when a subscription linked to an `Active` actor finishes, a think cycle is enqueued as a background job. Only an executed `CreateSubscription` action links a subscription to an actor, so an actor whose actions are proposed has no linked subscriptions and its cycles run on demand. **Cooldown.** An actor thinks at most once per `Beacon:AiActors:ThinkCooldownMinutes` (default `15`, `0` = off, at most `1440`; validated at startup), counted from its last think time. A cycle within the cooldown is skipped: a background one is only logged at Debug (actor id), and a subscription run within it is dropped, not deferred: nothing is enqueued for later. A cycle you request (**Think now**, or `POST .../think`) does **not** bypass it: it answers `success: false` with the minutes left. The cooldown starts when a cycle starts, so a failed cycle consumes it too; the initial setup of a new actor does not count towards it. The cycle claims the actor with one conditional update, so two hosts or job runs cannot both start one. A background cycle runs as the actor's creator. Before any LLM call or SQL, Beacon checks that the creator may still work ad hoc with the data source (Query in a project that links it). When the creator may not (disabled or archived, no longer a member of a restricted project, or only a Viewer), the cycle is recorded as `Skipped` with "Skipped: owner not authorized.", and the actor keeps its status. An actor without a recorded creator runs background cycles only while its data source is openly reachable, and never with `Beacon:Authorization:RequireProjectMembership: true`. ## Refining an actor Steer an actor with feedback on its edit page (**Refine actor**, field **Refinement feedback**) or with `POST /beacon/api/ai-actors/{id}/refine`: > "Focus on orders over $10,000 and check for duplicate payment attempts" The feedback is stored in the actor's conversation history and sent to the LLM with the schema and query context. Like a think cycle, the refinement records the actions the LLM chose as proposed, in a plan that keeps the feedback as its instruction (a revision of the plan carries it on). While the actor has a plan pending, a refinement is refused with `409` naming that plan, before anything reaches the LLM. ## Plans A **plan** is a stored bundle of analysis, key findings and proposed actions (for a query refinement: current SQL, proposed SQL and whether the query is locked) that a reviewer approves, rejects or sends back for revision. Approving a plan is the path that **executes** actions. For an actor that requires approval, the initial setup, a think cycle and a refinement that propose at least one action store them as one `PendingApproval` plan, saved together with the cycle's execution, whose analysis names the plan. A cycle that proposes nothing creates no plan. While the actor has a plan pending, it does not propose another: a think cycle is recorded as `Skipped` naming the pending plan and a refinement answers `409`, both before any LLM call. A plan is added only in a serializable transaction that first checks the actor has no plan pending, so two cycles, refinements or revisions finishing together never leave two pending plans. The one refused completes all the same: a think cycle or the initial setup keeps its proposals on its execution record, which names the pending plan; a refinement answers `409` naming it; a revision answers `409` saying the original plan was sent back but no revised plan was created. A revision for which the model proposes no action it can take creates no plan and fails. In the UI, the actor's page lists its pending plans under **Plans waiting for approval**. **Review** opens the plan: its analysis, findings and proposed actions with their SQL (a refinement shows the query's current SQL next to the proposed SQL), read-only, and its earlier versions with the feedback that shaped them. The dialog offers only the decisions the server says the caller may take: **Approve and run** (optional comment), **Reject** (reason required) and **Request revision** (feedback required); when the caller cannot approve, it says why. In the sidebar, **AI Actors** carries a badge with the number of plans waiting for the caller's decision. The same through the API: | Route | What it does | |---|---| | `GET /beacon/api/ai-actors/{id}/pending-plans` | The actor's plans awaiting review | | `GET /beacon/api/ai-actors/plans/{id}` | One plan with its analysis and actions, and what the caller may decide on it: `mayApprove` (with `approveBlockedReason` when not), `mayRejectOrRevise` | | `GET /beacon/api/ai-actors/pending-plans/count` | How many plans wait for the caller's decision: pending, not expired, of actors whose data source the caller may write; an Admin, or an Approver while the approval workflow is on, counts them all, anyone else only those of the actors they created | | `POST /beacon/api/ai-actors/plans/{id}/approve` | Approve (optional comment): the plan moves to `Executing`, its actions run, then `Executed`; if execution throws, the plan becomes `Rejected` | | `POST /beacon/api/ai-actors/plans/{id}/reject` | Reject (reason required); no action runs | | `POST /beacon/api/ai-actors/plans/{id}/request-revision` | Ask for changes (feedback required): a new plan is generated with an incremented version and a link to its parent | Plan statuses: `PendingApproval`, `Executing`, `Executed`, `Rejected`, `Expired`, `RevisionRequested`. Only a `PendingApproval` plan accepts a decision; the decision is a conditional update from that status, so a plan already decided (or decided by someone else at the same moment) answers `409`. A plan expires `Beacon:AiActors:PlanExpiryDays` days (default `7`, 1–365, validated at startup) after it was proposed. Expiry is computed when a plan is read, not written to the plan: an expired plan keeps the status `PendingApproval` in its row, but it is left out of `pending-plans`, approving, rejecting or revising it answers `409` saying it expired, and it no longer holds back the actor's next cycle. A plan read and each pending-plan summary carry `expiresAt`; a plan read also carries `isExpired`. The creator or an Admin decides a plan. While the [approval workflow](https://moberghr.github.io/beacon/security/approvals/) is enabled, approving takes a second person: an Approver (an Admin holds the permission) with write access to the data source, other than the creator and anyone who asked for a revision of the plan, unless the host allows self-approval (`ApprovalWorkflow.AllowSelfApproval`). ### Query change approvals When an approved plan's actions run, refinements of locked queries are skipped and recorded as failed actions, and the actor only touches queries and subscriptions it owns, within `MaxQueries` and `MaxSubscriptionsPerQuery`. Query SQL is validated without running it (the save checks and the engine's non-executing validation, such as `EXPLAIN` without `ANALYZE`). With the approval workflow enabled, a refined or created query's SQL is submitted for approval with the actor's creator as requester, and the plan's reviewer and revision requesters are recorded as contributors who cannot approve it; a subscription is created only for a query whose active version is approved for its exact content. See [Queries → Approval workflow](https://moberghr.github.io/beacon/features/queries/#approval-workflow). ## Cost tracking Each execution and plan records tokens used, estimated cost and the model; the actor keeps cumulative totals (`TotalTokensUsed`, `TotalCost`) shown on its page. `MaxQueries` and `MaxSubscriptionsPerQuery` bound what an actor may create. ## Troubleshooting **Actor is `Failed`.** Setup or a think cycle threw; the actor's last error has the message. Fix the cause (often the LLM configuration or data source connectivity), then resume the actor. **Think cycles show `Skipped`.** The creator may no longer work ad hoc with the data source. Restore their access, or have an Admin or UserAdmin change the owner. **"This AI actor thought recently".** The cooldown above; wait for the minutes the message names or change `Beacon:AiActors:ThinkCooldownMinutes`. **No automatic think cycles.** Background cycles follow runs of subscriptions linked to the actor, and an actor whose actions are proposed has none. Start cycles with `POST /beacon/api/ai-actors/{id}/think`. **"This plan is no longer pending approval".** The plan was already decided or revised. **"This plan expired before it was decided"**: it passed `Beacon:AiActors:PlanExpiryDays`. Fetch the actor's pending plans. **"This AI actor has a plan pending approval".** Decide that plan (or let it expire) before refining the actor; its think cycles are skipped meanwhile. ## Related features - [AI Integration](https://moberghr.github.io/beacon/features/ai-integration/): provider configuration and what reaches the provider - [Queries](https://moberghr.github.io/beacon/features/queries/): the query and versioning model - [Subscriptions](https://moberghr.github.io/beacon/features/subscriptions/): scheduling and delivery - [Approvals](https://moberghr.github.io/beacon/security/approvals/): the approval workflow --- # Security model > How Beacon protects your data end to end: identity and sessions, projects as the access boundary, functional permissions, four-eyes approvals, the single SQL execution gate, masking and LLM egress, secrets at rest, outbound notifications, the write path, deployment locks, audit and security events. Source: https://moberghr.github.io/beacon/security/ Beacon sits between people, AI assistants and production databases. Its security model assumes that any of them can make mistakes, and that an administrator's convenience should not quietly weaken a guarantee. This page is the map; each section links to the detail. ```mermaid flowchart TB subgraph Callers direction LR P[People
password · SSO · your sign-in] K[Programs
API keys] A[AI clients
API keys · Entra callers] J[Scheduled work
runs as its owner] end Callers --> ID[Identity & sessions] ID --> PA[Project access · one evaluator] PA --> GATE[SQL execution gate] GATE --> DB[(Your databases · read-only logins)] DB --> MASK[PII masking] MASK --> OUT[Caller · recipients · LLM] PA -.-> AUD[(Audit & security events)] GATE -.-> AUD OUT -.-> AUD ``` ## 1. Identity - **Sign-in paths:** Beacon's login form (PBKDF2-SHA256 passwords, 10 attempts per minute per IP, a 15-minute lockout after 5 wrong passwords), OIDC/SSO with tenant allow-list, guest blocking, required groups or app roles and optional MFA, bearer access tokens, API keys, or your own host's sign-in. → [Sign-in & identity](https://moberghr.github.io/beacon/integrate/authentication/) - **First run** needs a setup token; until an administrator exists nobody else can be provisioned. - **Break-glass:** with password sign-in off, at most three named accounts may still use a password; each use is logged as a warning (7119). - **MFA** (`RequireMfa`) covers the interactive SSO sign-in. Bearer tokens, API keys, MCP callers and break-glass passwords are not MFA-protected. Protect them with short lifetimes, project lists and alerting. ## 2. Sessions - The `Beacon.Auth` cookie is HttpOnly, SameSite=Lax and Secure on HTTPS; state-changing API calls need an antiforgery token. - Sessions are **revalidated on the server** every few minutes (user enabled, not archived, unchanged security stamp; roles reloaded) and end **12 hours** after sign-in however often they are renewed. - Changing a user's roles, password or status renews their security stamp and ends every session they have; realtime connections close with them. - The cookie's key ring (ASP.NET Core Data Protection) must be encrypted at rest outside Development, or the host does not start. ## 3. Projects are the access boundary One component, the project access evaluator, decides every access, for the UI, the API, MCP and scheduled work alike: - **Restricted** projects are visible only to members and Admins; everyone else gets **404**. New projects start restricted. - A **data source follows its projects**; anything that reads several data sources needs one project that links them all. - **API keys** are confined to the projects they name and capped by the lower of their scope and their owner's role, and never the Admin bypass. - **Scheduled work runs as its recorded owner**, re-checked right before every SQL statement or LLM call. If the owner may no longer do it, the run is recorded as skipped and is never run as the system. - **Grounding is project-scoped:** glossary terms, verified examples, learned patterns and evaluation cases of one project never reach another project's prompts, even when the projects share a data source. Every MediatR request declares how it is authorized, and tests fail when a new request does not. → [Authorization](https://moberghr.github.io/beacon/security/authorization/) ## 4. Separated duties The Admin role's duties are split into six **functional permissions** (Approver, Auditor, SecurityAdmin, UserAdmin, DataSourceAdmin, MigrationOperator), each grantable without any project data. Optional separation of duties refuses risky combinations (for example Approver + SecurityAdmin). A UserAdmin can never grant Admin or a functional role. API keys never hold a functional permission. → [Access control](https://moberghr.github.io/beacon/integrate/access-control/#3-functional-permissions) ## 5. Four-eyes approvals With the approval workflow on (opt-in), changes that decide what SQL runs or where data goes take effect only after a **second person** approves exactly what they reviewed: query changes, migration jobs, Custom SQL rules and notification destinations, and, when switched on, data-source connections and the LLM connection. Approvals are bound to a content hash, the requester cannot approve, and scheduled runs execute only reviewed SQL. → [Approvals](https://moberghr.github.io/beacon/security/approvals/) ## 6. The SQL execution gate Every SQL statement Beacon runs (MCP tools, previews, scheduled checks, saved-query tools, data-quality rules, AI features, value probes, migration reads and the final query of a cross-source join) passes **one gate** before it reaches a database: | # | Stage | What it does | |---|---|---| | 1 | Deployment lock | `Beacon:Mcp:ForceReadOnly` (default on) forces read-only whatever the caller asked. | | 2 | Length | SQL longer than the ceiling (100,000 characters by default) is refused. | | 3 | Characters | Control characters other than tab/LF/CR and Unicode non-characters are refused. | | 4 | Guardrail | Write and DDL patterns are refused, judged after the dialect's own tokenizer strips comments. | | 5 | AST validator | The SQL is parsed in the data source's dialect; anything but a read is refused. An unknown dialect is refused. | | 6 | Cross-database rule | Names qualified with another database, catalog or project; remote servers, files and storage; `USE`, `SET ROLE`, `EXECUTE AS`; SQL built from strings or variables are all refused. Always on, even with read-only off. | | 7 | System catalog rule | The engine's catalog, statistics and query-history relations (`pg_catalog` and `pg_…`, `information_schema`, `sys`, Snowflake `QUERY_HISTORY` and `RESULT_SCAN`, …) are refused; a source's tables and columns come from Beacon's metadata. Always on, even with read-only off. | | 8 | Host policy | For a host-exposed DbContext: only allow-listed tables and columns. | | 9 | Row cap | The cap is written into the SQL, overriding any bound the statement sets itself. Scheduled runs, previews and Beacon's own reads **fail** above the cap; MCP `query` and `ask` return the capped rows and say the result was truncated. Nothing is cut silently. | Then the statement runs on the data source's provider: | Engine | Database-level enforcement | |---|---| | PostgreSQL | `READ ONLY` transaction, exactly one statement per batch | | MySQL | `START TRANSACTION READ ONLY`, statement batches off | | SQL Server | A transaction that is always rolled back; a statement that ends it fails the run (7116) | | Synapse, Snowflake | None beyond the gate | | BigQuery | The connector's own parser check and a dry run whose statement type must be `SELECT` | | Databricks | The connector's own parser check and configured catalog | The gate is the primary control and database-level read-only is defence in depth, so **every data source needs a read-only login**, on every engine. Data sources whose engine cannot refuse writes carry a badge, and Beacon warns at startup. Every statement has a timeout: `Beacon:Query:StatementTimeoutSeconds` (120 s by default) for scheduled runs, previews and Beacon's own reads, and the project's MCP statement timeout (30 s by default) for MCP `query`, `ask` and the saved-query tools. Every statement also honours the caller's cancellation. ## 7. Masking and what leaves Beacon - **PII masking** (on by default, pinned by `Beacon:Mcp:ForcePiiDetection`) masks columns detected as PII (and their aliases, renames and wildcard expansions) in results that reach MCP clients, recipients and attachments. When Beacon cannot prove where a PII column went, it masks every column (fail closed). Host DbContext sources mask by output position. - **Database errors are redacted:** errors that can carry values (constraint violations, conversion failures) are reduced to their class; others keep their message with quoted values replaced. - **LLM providers receive schema, not rows**, unless a project turns on sample-value collection, value grounding or the eval judge (all off by default; `Beacon:Mcp:ForbidRowDataToLlm` pins them off). Every request that carries row-derived data is logged as event 7123 with the categories sent, never the values. - **Content retention:** a project (or the deployment, with `ForceNoContentRetention`) can keep MCP audit rows structural only (tool, tables, counts, timings, error class, keyed hashes), with no question, SQL or error text stored. A deployment that must keep that evidence pins retention on instead with `Beacon:Mcp:Audit:RequireContent`. ## 8. Secrets at rest - Data-source connection strings, tokens and recipient secrets are encrypted with **AES-256-GCM** (per-value nonce) under `Beacon:EncryptionKey`. The key is checked at startup in every environment and rotated through `Beacon:PreviousEncryptionKeys` and a re-encryption job. - API keys are stored as **SHA-256 hashes**; the raw key is shown once. - A host-exposed DbContext stores only the **name** of its connection string, never the string. - Secrets never reach logs; notification calls are excluded from HTTP tracing because their URLs can carry secrets. ## 9. Outbound notifications Notification destinations must be HTTPS on the channel's vendor hosts (Slack, Teams, Jira) or on public hosts (webhooks), unless `Beacon:Notifications` allows more. Private, loopback, link-local and cloud-metadata addresses are unreachable unless explicitly listed, and every connection is vetted at connect time. Only a SecurityAdmin (or Admin) manages recipients; new or changed destinations can require approval. → [Notifications](https://moberghr.github.io/beacon/features/notifications/) ## 10. One write path, off by default Beacon only writes to your data through **data migration**, which is off unless the host sets `Beacon:Migrations:Enabled`. Even then it writes only into a data source an Admin marked as a write target (PostgreSQL, MySQL or SQL Server; never a host-managed or read-only one), Beacon never reads from a write target, and with approvals on every job definition is reviewed by a second person. → [Data migration](https://moberghr.github.io/beacon/features/data-migration/) ## 11. Deployment locks Some settings are pinned by host configuration and bind Admins too. The UI shows them disabled and the API answers 409: | Key | Default | |---|---| | `Beacon:Mcp:ForceReadOnly` | `true` | | `Beacon:Mcp:ForcePiiDetection` | `true` | | `Beacon:Mcp:ForceNoContentRetention` | `false` | | `Beacon:Mcp:ForbidRowDataToLlm` | `false` | | `Beacon:Mcp:Audit:RequireContent` | `false` | | `Beacon:Mcp:Ceilings:MaxRowLimit` | none (SQL length 100,000 and question length 4,000 always apply) | | `Beacon:Migrations:Enabled` | `false` | | `Beacon:UserManagement:LocalLogin:*` | password sign-in on | Every MCP setting change is recorded in the settings history with who changed it. `Beacon:Profile: Regulated` requires the regulated values of these locks and of the audit, retention, row-data, ceiling, rate-limit and sign-in settings: it fills the keys a host left unset and fails startup, listing each, on any key set weaker; it never overrides a host value. A startup report (event 9130) names every weaker setting, also without the profile. → [Regulated profile](https://moberghr.github.io/beacon/getting-started/configuration/#regulated-profile-beaconprofile) ## 12. Audit and security events | Record | Where | Who reads it | |---|---|---| | **Security audit trail**: one append-only event per security-relevant action: sign-ins and sign-outs, users and roles, API keys, project access, data sources, delivery, settings, saved queries, approvals, ownership, alert tasks, curation, data contracts, migration jobs, and every read of audit data. Ids, codes and keyed digests only, no personal data or secrets | `SecurityAuditEvents` table; each event signed (HMAC-SHA256 under `Beacon:Audit:IntegrityKey`), verified on request, on the schedule the host sets (hourly in the sample host) and before a purge; kept forever unless `Beacon:Audit:RetentionDays` is set; copied to the `Beacon.Audit` log stream (9104) once committed | Auditors, on the **Security Audit** page or through `GET /beacon/api/security-audit` (each read itself recorded) | | **MCP audit log**: one row per tool call, refusals and failures included: tool, caller, project, data source, rows, timing, trace id, MCP session id, a keyed hash of the parameters, and every statement the call ran on a data source (its SQL where the project retains content) | `McpAuditLog` and `McpAuditStatement` tables; optional fail-closed mode (`Beacon:Mcp:Audit:Required`); retention via `RetentionDays` | Auditors, through `GET /beacon/api/mcp/audit` (each read recorded in the security audit trail) | | **Settings history**: admin settings, MCP settings, migration write-target changes, with old and new value (secrets as `***`) | `AppSettingHistory` table | Auditors and SecurityAdmins | | **Security log events**: structured log events 7101 to 7123 with ids only, no personal data | Your log pipeline. The log events themselves are not stored; the actions most of them report are also recorded in the security audit trail. Refused and hidden reads (7108, 7109), skipped scheduled work (7111), read-only transaction ends (7116), key-ring and re-encryption notices (7121, 7122) and row data sent to the LLM (7123) are on the log only | Your SIEM | Queries, subscriptions, recipients, data contracts and migration jobs also record who created them and who changed them last; an Auditor or a UserAdmin lists what a user owns, created or changed last. → [Security audit trail](https://moberghr.github.io/beacon/security/audit-trail/) | Event | Id | Event | Id | |---|---|---|---| | MemberAdded | 7101 | UserRolesChanged | 7113 | | MemberRoleChanged | 7102 | UserCreated | 7114 | | MemberRemoved | 7103 | MigrationWritesChanged | 7115 | | AccessModeChanged | 7104 | ReadOnlyTransactionEnded | 7116 | | DataSourcesLinked | 7105 | UserSessionsRevoked | 7117 | | RepositoryScanRequested | 7106 | SessionEnded | 7118 | | RepositoryTokenChanged | 7107 | BreakGlassSignIn | 7119 | | AccessRefused | 7108 | PasswordSignInRefused | 7120 | | AccessHidden | 7109 | DataProtectionKeyRing | 7121 | | DataSourceUnlinked | 7110 | StoredValuesReEncrypted | 7122 | | ScheduledWorkSkipped | 7111 | RowDataSentToLlm | 7123 | | ProjectCreatedOpen | 7112 | | | Alert at least on 7119 (break-glass), 7113/7114 (role changes, new users), 7115 (write targets) and 7116 (a statement tried to end SQL Server's read-only transaction), and on any increment of the counters `beacon.security_audit.write_failures` (a security audit event was not stored) and `beacon.security_audit.verification_failures` (stored events failed verification). → [MCP observability & audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/) ## Your part Beacon's guarantees assume the host does its share: - read-only logins on every data source; - secrets in a secret store, Data Protection keys encrypted at rest; - `UseForwardedHeaders` with known proxies, and `Beacon:PublicBaseUrl` set, behind a reverse proxy; - HSTS and `AllowedHosts` set by the host, and Beacon's origin check switched to `Enforce` once its warnings are gone; - the middleware order from [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/#the-middleware-pipeline); - every job scope marked as system work; - security log events and the `Beacon.Audit` category (including the security audit trail's copies and verification checkpoints, 9104 to 9114) shipped to a monitored log store; - `Beacon:Audit:IntegrityKey` set from a secret store, and the security audit verification and retention jobs scheduled (the sample host schedules both). The [production checklist](https://moberghr.github.io/beacon/integrate/production/) collects all of these. --- # Authorization > How Beacon decides access: projects as the boundary, global and project roles, project actions, functional permissions with a route matrix, separation of duties, identity-provider role mapping, 404 versus 403, scheduled work and shared data sources. Source: https://moberghr.github.io/beacon/security/authorization/ Beacon authorizes in two layers. A caller's **global role** (read, write) comes from a pluggable `IBeaconAuthorizationProvider`. Its **project access** (which projects, and through them which data sources, it may use, and which action it may take there) comes from one evaluator, `IProjectAccessEvaluator`. On top of both, the Admin role's duties are split into six **functional permissions** that grant no project data. This page is the reference for those rules. For accounts, sign-in and sessions see [User management](https://moberghr.github.io/beacon/security/user-management/); for how the controls fit together see [Security model](https://moberghr.github.io/beacon/security/). ## Overview | Concept | Values | Decided by | |---|---|---| | Global role | Admin (level 3), Editor (2), Viewer (1); functional roles are level 0 | `IBeaconAuthorizationProvider` (`DatabaseAuthorizationProvider` with user management) | | Project role | Viewer, Editor, ProjectAdmin | `ProjectMember` rows, or a host `IProjectMembershipSource` | | Project action | Read, Run, Query, Write, Manage, ManageMembers, Curate | `IProjectAccessEvaluator` | | Functional permission | Approver, Auditor, SecurityAdmin, UserAdmin, DataSourceAdmin, MigrationOperator | live stored roles, or an identity-provider app role mapped at sign-in | Authorization, user management and the login form are all **off by default** (`options.Authorization.Enabled`, `options.UserManagement.Enabled`, `options.Authentication.EnableLoginForm`). With user management on, the stored roles cap every signed-in caller even when `Authorization.Enabled` is `false`; only with both off is every signed-in caller an Editor. The building blocks: - `IBeaconUserContext`: the current caller (`UserId`, `UserName`, `DisplayName`, `Email`, `IsAuthenticated`, `HasClaim`). - `IBeaconAuthorizationProvider`: the caller's global read and write permission. - `IProjectAccessEvaluator`: the caller's project access per action. - `IBeaconPermissionAccessor`: the functional permissions the caller holds now. - `IBeaconAuthenticationProvider`: how a login-form sign-in is checked (the sample host uses `DatabaseAuthenticationProvider`). ## Enabling authorization With Beacon's [user management](https://moberghr.github.io/beacon/security/user-management/), roles come from Beacon's database and `DatabaseAuthorizationProvider` is registered for you: ```csharp builder.Services.AddBeaconServices(builder.Configuration, options => { options.AddBeaconScheduler(); options.Authorization.Enabled = true; options.Authentication.EnableLoginForm = true; options.AddAuthenticationProvider(); options.EnableUserManagement(); }) .UsePostgreSql(); ``` A host that signs users in itself registers its own provider (`options.AddAuthorizationProvider()`, see [Custom authorization provider](#custom-authorization-provider)) and a claims transformation that adds Beacon's claims, passed to `AddBeaconHostInfrastructure()`. The full wiring is on [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/) and [Access control](https://moberghr.github.io/beacon/integrate/access-control/). ```csharp public sealed class MyClaimsTransformation : IClaimsTransformation { public Task TransformAsync(ClaimsPrincipal principal) { var identity = (ClaimsIdentity)principal.Identity!; var role = principal.IsInRole("Administrator") ? "Admin" : principal.IsInRole("PowerUser") ? "Editor" : "Viewer"; identity.AddClaim(new Claim(BeaconClaims.Role, role)); identity.AddClaim(new Claim(ClaimTypes.Role, role)); // the Admin checks read this one identity.AddClaim(new Claim(BeaconClaims.UserId, principal.FindFirstValue(ClaimTypes.NameIdentifier)!)); identity.AddClaim(new Claim(BeaconClaims.UserName, principal.Identity!.Name!)); return Task.FromResult(principal); } } builder.Services.AddBeaconHostInfrastructure(); ``` Beacon's claim types: `BeaconClaims.UserId` (`beacon:user_id`), `UserName` (`beacon:user_name`), `Role` (`beacon:role`), `Permission` (`beacon:permission`). ### Admin role claim Admin-only routes (the `BeaconApiAdmin` policy) and the rules that let an Admin act on what another user owns ask who the caller is in Beacon. While user management is on, a stored user is an Admin by the roles stored **now**, not by any claim the session carries: a demoted Admin loses Admin routes and the Admin bypass on the next request, and a session without a stored user is never an Admin and reaches no project at all. So when a host signs people in (`HostOwnsSignIn`), a role claim its own sign-in adds, such as an Entra app role named `Admin`, never makes anyone Beacon's Admin. On a host without Beacon user management, the standard `ClaimTypes.Role` claim with the value `Admin` is Beacon's Admin; `beacon:role` alone is not enough. Beacon's own login and OIDC sign-in emit it from the user's roles. An API key never carries a role, so it is never an Admin. ## Global roles | Role | Level | Read | Write (create, edit, run ad hoc) | |---|---|---|---| | **Admin** | 3 | Yes | Yes | | **Editor** | 2 | Yes | Yes | | **Viewer** | 1 | Yes | No | | Functional roles (Approver, Auditor, …) | 0 | No | No | The provider answers only read (level 1 and up) and write (level 2 and up, or a super admin). There is no delete level: deleting and archiving are the project action **Manage** (see [Project roles](#project-roles)), which an Editor holds in an open project and as the recorded owner. A super admin (`IsSuperAdmin`, the first-run account) passes every check. When `Authorization.Enabled` or user management is on, `/beacon/api` enforces the global role: `GET` needs read permission and every other method needs write permission. A Viewer may also run a stored query's preview (a draft's preview needs write permission), create Read API keys, revoke their own keys and change their own password. A route of a [functional permission](#functional-permissions) admits its holder instead. ## Project access Projects are the authorization boundary. A caller reaches a project's queries, subscriptions, stored runs, tasks, data contracts, AI actors, documentation and data-source metadata only through a project it may use, and a data source only through a project that links it. `IProjectAccessEvaluator` decides this once per request for every caller; the REST API, the UI, the MCP playground and MCP tool calls all consult it. ### Open and restricted projects - **Open**: everyone with a Beacon role uses the project at their global role. A Viewer reads and runs saved SQL; an Editor also writes and works ad hoc. - **Restricted** (`requiresMembership`): only the project's members and Admins use it; to anyone else it does not exist (`404`). - **New projects are created restricted.** When the creator is an active Beacon user and Beacon manages project membership, the project starts restricted with the creator as its project admin (an Admin who creates a project becomes its project admin too). A creator without an active Beacon user, or any creator while the host manages membership or membership is unavailable, gets an open project (nobody could be a member of a restricted one), and Beacon logs `ProjectCreatedOpen` (7112). The `POST /beacon/api/projects` answer carries `requiresMembership`. - **Restricting and opening.** An Admin or a UserAdmin restricts a project with the **Access** card on the project page (`PUT /beacon/api/projects/{id}/access` with `{ "requiresMembership": true }`). Opening a restricted project is an Admin's alone (`403` for anyone else). Restricting answers `409` while project membership is unavailable. Add members (a project admin first) before restricting. The card lists the project's data sources that another open project also links (they stay reachable through it) and every other project that shares a data source with it. - **`Beacon:Authorization:RequireProjectMembership: true`** restricts every project, whatever its own setting, and makes data sources no project links Admin-only. It needs project memberships (Beacon user management or a host [membership source](#host-seams)); the host fails at startup otherwise. - **Archived projects** grant nothing. No route archives a project. ### Project roles A member holds one role per project. A global Viewer is a Viewer in every project, whatever their membership; Editors and Admins act with their membership role (and in an open project with the higher of that and their global role). | Action | Viewer | Editor (restricted) | Editor (open) | Project admin | Admin | |---|---|---|---|---|---| | **Read**: resources, SQL, metadata, stored results, documentation | ✓ | ✓ | ✓ | ✓ | ✓ | | **Run**: execute saved SQL, evaluate data contracts | ✓ | ✓ | ✓ | ✓ | ✓ | | **Query**: ad-hoc and draft SQL, LLM work (playground, AI actors) | – | ✓ | ✓ | ✓ | ✓ | | **Write**: create and change resources | – | ✓ | ✓ | ✓ | ✓ | | **Manage**: delete, archive, take over a lock, force a metadata refresh | – | as recorded owner | ✓ | ✓ | ✓ | | **ManageMembers**: manage members, link data sources | – | – | – | ✓ | ✓ | | **Curate**: review learned patterns, edit or regenerate documentation, apply or reject documentation patches, verify relationships and delete verified ones | – | – | – | ✓ | ✓ | | Restrict the project | – | – | – | – | ✓ (also UserAdmin) | | Open a restricted project | – | – | – | – | ✓ | The recorded owner is a data contract's or subscription's owner, an AI actor's creator and a query's lock holder. Owner rules on tasks, contracts and AI actors apply on top: both the project action and the owner rule must pass. A UserAdmin also manages the members of every project and restricts any project without being its member; a DataSourceAdmin links data sources no open project holds (see [Functional permissions](#functional-permissions)). **ManageMembers and Curate are membership-only.** An open project never grants them, and they are never granted to an API key (Read or Execute scope), an MCP caller (user or system) or the owner of scheduled work. Curation that records who decided (reviewing a learned pattern, verifying a relationship) also needs a signed-in Beacon user (`403` otherwise). A data source's shared grounding (relationships, documentation patches to descriptions) needs Curate in **every** live project linking the source, or the DataSourceAdmin permission; a relationship someone without Curate creates or relabels is stored unverified until a curator verifies it. Golden examples and the glossary stay Admin-only. See [Knowledge base → Who curates grounding](https://moberghr.github.io/beacon/mcp/knowledge-base/#who-curates-grounding). ### Data sources follow their projects - A data source is reachable only through a project the caller may use. A resource that reads several data sources is reachable only through **one** project that links all of them. Resources addressed by their data sources alone (an ad-hoc preview, a metadata read, a data contract) have no home project: any such project serves. - **Unlinked data sources.** A data source no project links is reachable as before only if it existed before project access was introduced (`OpenWhenUnlinked`). A new data source is Admin-only until it is linked: link it when you create it (the **Projects** field of the **New data source** dialog, `projectIds` on `POST /beacon/api/data-sources`). Removing a source's last link makes it Admin-only again. - **Shared data sources.** A source linked to an open project is reachable through that project by everyone with a role, even when a restricted project links it too. Writes to a source's shared state (relationships, description edits, a forced metadata refresh) need Write in **every** live project that links it (or DataSourceAdmin, or Admin), because that state grounds every project using the source. - **Linking.** A project admin (or an Admin) links data sources into the project with **Link data sources** (`POST /beacon/api/projects/{id}/data-sources`). Anyone but an Admin may link only sources already linked to a live open project; a host's data source links only into the project the host declares for it, for Admins too. An unreadable source answers `404`, a readable one that may not be linked `403`, and nothing is linked partially. - **Queries without steps** read no data source, so no project holds them: their creator (the query's owner) and Admins reach them. - **One database login per boundary.** Project access decides who may run SQL through Beacon, not what the database login can read. Give each data source a login that sees only what the projects linking it should see. Views, synonyms and functions in the source's own database read with that login's grants. ### Database names in SQL SQL that Beacon runs against a data source may name objects of that source's own database only. A name qualified with a database (`database.schema.table` on SQL Server, Azure Synapse, PostgreSQL and Snowflake, `database.table` on MySQL and SQLite, `catalog.schema.table` on Databricks, `project.dataset.table` on BigQuery) must carry the database, catalog or project the data source's connection is configured for; anything else is refused before the SQL runs. Unqualified and schema-qualified names are always allowed. The check applies whether or not read-only enforcement is on. - **The configured database** comes from the connection string (SQL Server and Synapse `Initial Catalog`/`Database`, PostgreSQL `Database`, MySQL `Database`, Snowflake `db`; SQLite is `main`) or the connector settings (Databricks `Catalog`, BigQuery `ProjectId`). A data source without one accepts no database qualifier at all: set it in the connection, or write `schema.table`. When the settings cannot be read, no qualifier is accepted either. - **Comparison** follows the engine: PostgreSQL folds unquoted names to lower case and Snowflake to upper case; MySQL compares exactly; SQL Server, SQLite, Databricks and BigQuery ignore case. - **System databases are other databases** (MySQL `information_schema`, SQL Server `master`, Snowflake `SNOWFLAKE`, Databricks `system`). - **The engine's catalog is refused too**, also inside the source's own database and in `DESCRIBE`: system schemas (PostgreSQL `pg_catalog` and every `pg_…` schema, `information_schema` on every engine that has one, SQL Server `sys`, MySQL `mysql`, `performance_schema` and `sys`, Snowflake's `SNOWFLAKE` database, BigQuery `INFORMATION_SCHEMA`), relations named for the catalog (PostgreSQL `pg_…` in any schema, SQL Server compatibility views such as `sysobjects` unqualified or in `dbo`, BigQuery `__TABLES__`, SQLite `sqlite_…`, `pragma_…` and `dbstat`), system table functions (SQL Server `fn_dblog`, `fn_get_sql` and the like; Snowflake `QUERY_HISTORY…`, `LOGIN_HISTORY…`, task and copy history, `VALIDATE`, `RESULT_SCAN`, `GET_QUERY_OPERATOR_STATS`), and PostgreSQL `pg_stat…` and `pg_get_…` functions and `obj_description`, `col_description` and `shobj_description`. Catalog statistics, definitions and history hold column values and other sessions' statements. On PostgreSQL, `ONLY` before a table name is refused too (quote a table named `only`), and so is a table of your own named `pg_…`. A source's tables and columns come from Beacon's metadata instead. The gate code is `system_catalog`. - **Also refused**, with read-only enforcement on or off: names with more parts than the full form (SQL Server linked-server names); names resolved only at run time (`IDENTIFIER(...)`, Snowflake `TABLE('...')`, `GET_DDL`, `OBJECT_DEFINITION`); functions and procedures that reach other servers, files or storage (`dblink…`, `OPENROWSET`, `OPENQUERY`, `EXTERNAL_QUERY`, `read_files`, PostgreSQL `COPY … PROGRAM` and server-file `COPY`, Databricks network and stream functions and file paths); database, role and user switches (`USE`, `SET ROLE`, `EXECUTE AS`, `set_config`, SQLite `ATTACH`); SQL run from a string, variable or session (`EXEC @sql`, `sp_executesql`, `EXECUTE IMMEDIATE`, `PREPARE`/`EXECUTE`, PostgreSQL `DO` and string-bodied functions); and SQL Beacon cannot parse. - **Objects in the source's own database** (views, synonyms, foreign tables, procedures) are allowed by name and read with the login's grants. Grant the login only what the data source should expose. - **Where:** every statement Beacon runs on a data source passes the same execution gate: the MCP tools, saving a query or step (also by an AI actor), previews, subscription and test-delivery runs, AI actor checks, data-migration reads, Custom SQL rules, documentation enum discovery and value-grounding probes. The in-memory join of a multi-step query reaches no database and is not checked. `IDataSourceProvider.Execute*` called directly by a host, and the metadata extractors (fixed catalog SQL), are not checked. See [Security model](https://moberghr.github.io/beacon/security/) for the full execution gate. ### Guardrails of shared data sources A data source that several live projects link is held to the strictest guardrails among them, whichever project's caller queries it: read-only enforcement and PII detection are on when any project turns them on; value grounding and sample-value collection are off when any turns them off; every project's custom PII patterns apply; and the row limit and the value-grounding probe budget are the lowest any of them sets. The statement timeout, result size and plan cost settings are resolved the same way (lowest wins) and applied by the MCP tools. A source only one project links keeps that project's settings; archived projects count for nothing. Read-only enforcement, PII detection and the row limit apply to the MCP tools that run SQL (`query`, `dry_run`, `ask`, saved-query tools, and so the playground); each statement they run takes the project's MCP statement timeout. The grounding settings apply to grounding (`ask`, `get_query_context`) and to sampling values during a metadata refresh, which collects only while collection is on for **every** live project linking the source. Stored sample values pass the PII screen again whenever they are shown. For an Admin, the project detail lists the effective value and the deciding project per shared source (`sharedSourceGuardrails`). ### Who the caller is | Caller | Project access | |---|---| | Admin (enabled user with the Admin role, or a super admin) | every live project, every action | | Signed-in Beacon user | open projects at their global role; restricted projects through membership | | Disabled or archived user | nothing | | Session without a stored user while user management is on (a host principal, a bearer token not bound to a user) | nothing | | Session on a host without Beacon user management | open projects at the global role; the `Admin` role claim counts; restricted projects are Admin-only unless the host supplies a [membership source](#host-seams) | | API key | its owner's access within its project list, capped by the lower of the key's scope and the owner's global role; never the Admin bypass, never ManageMembers or Curate. A key without a list (created before lists were required) has its owner's access | | Mapped Entra system caller on MCP | its configured projects, for reading, running saved SQL and ad-hoc queries | | Mapped Entra user caller on MCP | its configured projects within its Beacon user's access; without a Beacon user, its configured open projects | | Browser session on `/beacon/mcp` | refused with `403` (browsers use the MCP playground) | | Scheduled work done for its owner | its owner's access at that run (see [Scheduled work runs as its owner](#scheduled-work-runs-as-its-owner)) | | Scheduled work without an owner (created before owners were recorded) | openly reachable data sources only; nothing with `RequireProjectMembership` | | Other work without a request, in a scope marked as system work | the system, unrestricted | | Work without a request, in an unmarked scope | nothing (fail closed, a warning is logged) | See [API keys](https://moberghr.github.io/beacon/security/api-keys/#project-restrictions), [MCP server](https://moberghr.github.io/beacon/mcp/server/) and [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/) for their own rules. ### How it is enforced: 404 versus 403 - **Every request declares its check.** A request that addresses a resource by id implements `IProjectAuthorizedRequest` and names the resource and action; a MediatR pipeline behaviour resolves the resource's project (or data sources) and refuses before the handler runs. A list request implements `IProjectFilteredRequest` and filters in SQL, so totals and pages never count what the caller cannot see. A request with nothing project-scoped is marked `[ProjectScopeExempt("reason")]`. Tests fail when a request's id property is neither checked nor exempt. - **404 or 403.** A resource that is missing, or that the caller may not read, answers `404`. The caller cannot tell the two apart. A resource the caller may read but whose action is not granted answers `403`. Refusals throw `BeaconNotFoundException` and `BeaconForbiddenException`; on `/beacon/api` they become RFC 7807 problem responses without the message, in every host. `BeaconConflictException` and `BeaconRateLimitedException` map to `409` and `429`, and `BeaconValidationException` (input over a size limit) to `400` with its message. Any other exception, including `UnauthorizedAccessException`, is left to the host. - **In-process requests are checked too.** The behaviour runs on every `IMediator.Send` made while an HTTP request is served, so a host endpoint that sends Beacon requests is checked against that request's caller. - **Logged.** A `403` logs `AccessRefused` (7108), a hidden resource `AccessHidden` (7109), with ids, the reason, the principal kind and the caller's user id. The events never carry names or values. - **The UI** hides what a caller may not do (the **Members** tab, **Link data sources**, the **Access** card) and shows a resource it may not read as not found. - **MCP.** Tool calls over `/beacon/mcp` and in the playground take the caller's projects from the evaluator on every call: Read for the catalog tools, Query for `ask`, `query`, `dry_run` and `feedback`. Access that cannot be determined answers a retry message and fails `tools/list`, never a shorter list. ### Members A project admin, a UserAdmin or an Admin manages members on the project page's **Members** tab. Search finds people by **display name** only (case-insensitive, at least two characters, up to 20 matches, rate-limited); people are shown by display name or as "User {id}", never by user name, e-mail address or external id. A disabled or archived member stays listed as inactive and grants nothing. A project keeps an **effective project admin**: an enabled, non-archived member with the project admin role whose global role is Editor or above. Removing or demoting the last one is refused (`400`) unless an Admin does it. The check and the change run in one serializable transaction; a collision answers `409`. | Route | Purpose | |---|---| | `GET /beacon/api/projects/{id}/members` | members, paged | | `GET /beacon/api/projects/{id}/member-candidates?search=` | people to add | | `PUT /beacon/api/projects/{id}/members/{userId}` | add a member or change their role (`{ "role": 1 \| 2 \| 3 }`: Viewer, Editor, ProjectAdmin) | | `DELETE /beacon/api/projects/{id}/members/{userId}` | remove a member | | `PUT /beacon/api/projects/{id}/access` | restrict (Admin, UserAdmin) or open (Admin) | | `POST /beacon/api/projects/{id}/data-sources` | link data sources (`{ "dataSourceIds": [...] }`) | | `DELETE /beacon/api/projects/{id}/data-sources/{dataSourceId}` | unlink a data source (Admin) | When the host manages membership, the membership routes answer `409` ("Project membership is managed by the host."); when membership is unavailable, `409` ("Project membership is not available in this deployment."). The Members tab is hidden in both cases. ### Unlinking data sources An Admin removes a data source's link to a project with `DELETE /beacon/api/projects/{id}/data-sources/{dataSourceId}`, or from the **Access** card of a project that shares the source. Members lose the source through that project at once; other projects that link it keep it. When the removed link was the last, the source becomes Admin-only. A data source never becomes unlinked and open. A host's data source cannot be unlinked (`409`). Each unlink is logged as `DataSourceUnlinked` (7110). ### Scheduled work runs as its owner A subscription's scheduled run, a data contract's scheduled evaluation and an AI actor's background think cycle run as the owner they recorded (the subscription's or contract's owner, the actor's creator) and are re-authorized at **every** run, right before any SQL or LLM call: | Work | The owner needs | When refused | |---|---|---| | Subscription run | Run in one project that holds every data source of the query's steps | the run is recorded as **Skipped** with "Skipped: owner not authorized."; no SQL, no delivery; flagged `ownerNotAuthorized` | | Data contract evaluation | Run in a project that links the contract's data source | nothing is evaluated or alerted; the contract records when skipping started and is flagged until an evaluation runs again | | AI actor background cycle | Query in a project that links the actor's data source | the cycle is recorded with phase **Skipped** | A refused run is not a failure and is never run as the system; it is logged as `ScheduledWorkSkipped` (7111). Work without an owner runs only while every data source it reads is openly reachable, and never with `RequireProjectMembership`. On a host without Beacon user management, work whose owner has no Beacon user runs in the system scope and Beacon logs a warning naming the owner; turn on Beacon user management to have each run checked against its owner. Work a request asks for runs as the request's caller and answers `403` when the caller may not. This covers running a subscription (`POST /beacon/api/subscriptions/{id}/execute`), **Evaluate now** on a contract, and a think cycle (`POST /beacon/api/ai-actors/{id}/think`). An **Admin or a UserAdmin** changes the owner with `PUT /beacon/api/subscriptions/{id}/owner`, `PUT /beacon/api/data-quality/contracts/{id}/owner` or `PUT /beacon/api/ai-actors/{id}/owner` (`{ "userId": 12 }`); a UserAdmin never makes themselves the owner. The new owner must be an enabled user who may do the work (Run for a subscription or contract, Query for an AI actor), or the change answers `409`. ## Functional permissions The Admin role's duties are split into six functional permissions. An Admin holds all six; a user who should do one duty, and not the others, gets that permission's role instead. | Permission | The duty | |---|---| | **Approver** | Approves and rejects query changes and governed changes (migration jobs, Custom SQL rules, recipient destinations and, when covered, data-source connections), never their own; reads every project's approval requests except the LLM connection's, which only an Admin reads and decides. See [Approvals](https://moberghr.github.io/beacon/security/approvals/) | | **Auditor** | Reads and verifies the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/); reads the MCP audit, the MCP and project settings, the admin settings (secrets masked), every user's API keys and what a user owns, created or changed last | | **SecurityAdmin** | Changes the MCP settings and the admin settings except the LLM connection and base URL; manages recipients; publishes saved queries as MCP tools; sets repository access tokens | | **UserAdmin** | Manages users (external users only) and their roles and sessions, sees what a user owns, created or changed last, revokes any user's API keys, changes the owner of scheduled work, manages the members of any project, restricts projects | | **DataSourceAdmin** | Creates, tests, edits (replacing the connection), disables, enables and deletes data sources; links a data source no open project holds into a project it administers, when it can read the source; changes shared data-source state; forces a metadata refresh without the cooldown | | **MigrationOperator** | Runs [data migration](https://moberghr.github.io/beacon/features/data-migration/) for the jobs whose data sources it may query and write (mapped only while `Beacon:Migrations:Enabled`) | **Admin only:** the glossary and eval/golden curation, the LLM provider, endpoint, region, authentication mode and credentials, the base URL, turning read-only enforcement or PII detection off (only where the deployment unlocks them, see [MCP safety locks](#mcp-safety-locks)), opening a restricted project, unlinking a data source, marking a migration write target, creating data contracts with Custom SQL rules, and the sample host's Warp dashboard. ### Route and permission matrix Paths are under `/beacon/api`. "Requires" means the route's policy is the permission (an Admin passes too); "Also admits" means the route is open to everyone with read access and additionally to the permission's holder without a Viewer or Editor role. | Route | Requires | Also admits | |---|---|---| | `POST /approvals/{id}/approve`, `/approvals/{id}/reject` | Approver | | | `POST /approvals/changes/{id}/approve`, `/approvals/changes/{id}/reject` | Approver | | | `GET /approvals/pending`, `/approvals/{id}`, `/approvals/changes`, `/approvals/changes/{id}` | | Approver | | `GET /mcp/audit` | Auditor | | | `GET /mcp/settings`, `/mcp/settings/locks`, `/mcp/projects/{projectId}/settings` | Auditor or SecurityAdmin | | | `GET /admin-settings`, `/admin-settings/llm-providers` | Auditor or SecurityAdmin | | | `GET /api-keys/admin` | Auditor or UserAdmin | | | `PUT /mcp/settings`, `PUT /mcp/projects/{projectId}/settings` | SecurityAdmin | | | `PUT /admin-settings`, `POST /admin-settings/test-llm` | SecurityAdmin (LLM connection and base URL: Admin, checked by the handler) | | | `POST /recipients`, `PUT /recipients/{id}`, `DELETE /recipients/{id}` | SecurityAdmin | | | `PUT /queries/{id}/mcp-tool` | SecurityAdmin | | | `PUT /projects/repositories/{id}/token` | SecurityAdmin | | | `/users/*` (list, roles, create, update, toggle, `PUT /users/{id}/roles`, sign-out-everywhere) | UserAdmin | | | `DELETE /api-keys/admin/{id}` | UserAdmin | | | `PUT /subscriptions/{id}/owner`, `/data-quality/contracts/{id}/owner`, `/ai-actors/{id}/owner` | UserAdmin | | | `PUT /projects/{id}/access` | UserAdmin (opening: Admin, checked by the handler) | | | `GET /projects/{projectId}/members`, `/projects/{id}/member-candidates`, `PUT`/`DELETE /projects/{id}/members/{userId}` | | UserAdmin | | `POST /data-sources`, `POST /data-sources/test-connection`, `PUT /data-sources/{id}`, `PUT /data-sources/{id}/enabled`, `DELETE /data-sources/{id}` | DataSourceAdmin | | | `/migrations/*` | MigrationOperator | | | `GET /home/migration-summary` | | MigrationOperator | | `/glossary/*`, `/eval/*`, `PUT /data-sources/{id}/migration-writes`, `DELETE /projects/{id}/data-sources/{dataSourceId}` | Admin | | The complete route list is on [Routes & endpoints](https://moberghr.github.io/beacon/reference/routes/). ### Rules - **Functional roles grant no project data.** A functional role is a level-0 role with neither read nor write permission, so a user holding only functional roles reads no project, data source or resource and gets `403` elsewhere. Give such a user a Viewer or Editor role as well to let them use projects. Two duty-specific exceptions: an Approver reads every project's approval requests with their SQL; an Auditor reads every project's MCP audit rows. - **Permissions are read live.** Each route carries the policy `BeaconApi.{Permission}`, which admits the Admin role or the permission's role among the session's claims; the permission filter then admits the caller only if they hold the permission **now** (`IBeaconPermissionAccessor`). A user whose role was removed gets `403` although their session still carries the claim. The `BeaconApi.` policy-name prefix is reserved for Beacon. - **API keys hold none.** A key carries no role and no functional permission, whatever its owner holds. A key refused on such a route is told: *"API keys cannot use administrative routes; sign in as a user with the X permission."* - **A UserAdmin cannot raise anyone, itself included.** A UserAdmin who is not an Admin never assigns or removes the Admin role or a functional role, never changes their own account, roles or project membership, and never touches a **protected user**: an Admin, a super admin, or a holder of any level-0 system role (other UserAdmins included). A UserAdmin cannot create an **internal** (password) user nor change a user's user name or e-mail address (`403`), so it cannot make an account it can sign in to. No route sets another user's password. A UserAdmin restricts projects but does not open one, and never makes themselves an owner of scheduled work. - **What a UserAdmin can do, and how to watch it.** A UserAdmin creates external users, gives them Viewer or Editor, and adds unprotected users to any project. The identity provider and its [admission rules](https://moberghr.github.io/beacon/security/user-management/#single-sign-on-admission) bound what an external user reaches. To review these changes, alert on events **7114** (`UserCreated`), **7101** (`MemberAdded`) and **7113** (`UserRolesChanged`) whose actor holds UserAdmin. - **A MigrationOperator has no bypass.** Creating, running and deleting a job need Query on the declared source and on every step's data source and Write on the destination, and the destination may not be more widely reachable than its sources. See [Data migration](https://moberghr.github.io/beacon/features/data-migration/). - **A DataSourceAdmin links without gaining access.** Linking gives no membership of other projects; the DataSourceAdmin still needs the project admin role in the target project. A source it cannot read is linked only by someone who can (in practice an Admin). - **A SecurityAdmin tightens, an Admin loosens.** A SecurityAdmin turns read-only enforcement and PII detection on, globally and per project; turning either off is an Admin's (`403`), and nobody's while the deployment locks them (`409`). Changing where a recipient delivers is refused (`403`) when the recipient is attached to work the SecurityAdmin cannot read. - **The authorization provider can veto.** A host may refuse a permission with `IBeaconAuthorizationProvider.AllowsFunctionalPermissionAsync` (a default member that returns `true`). `GET /beacon/api/auth/permissions` returns the caller's `canRead`, `canWrite`, `isAdmin` and `permissions` (the names held now; an Admin's lists all six). The shell uses it to show the pages each permission opens. It also carries `build` (the running release `version` and source `commit`), which the signed-in sidebar shows; the web bundle itself carries no build identity, so nothing about the deployed build is readable before sign-in. ### Holding a permission - **Through a Beacon role.** Each permission is granted by one level-0 system role, named after it by default; `Beacon:Authorization:Roles:{Permission}` maps it to another name (see [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/#functional-permissions-role-names-identity-provider-app-roles-separation-of-duties)). Beacon creates the roles when roles are listed for assignment (`GET /beacon/api/users/roles`), at first-run setup and at SSO provisioning. A role someone created earlier under one of these names is left alone and grants nothing (seeding logs its id). Assign roles on the **Users** page or with `PUT /beacon/api/users/{id}/roles` (`{ "roleIds": [...] }`). - **Through an identity-provider app role** (opt-in, `Beacon:Authorization:IdpRoles`). At single sign-on, app roles listed in `IdpRoles:Map` add their permission to the session's claims, issued by Beacon's sign-in. Nothing is stored; the permission counts for `IdpRoles:ClaimLifetimeMinutes` (default 60) after that sign-in, does not show on the Users page, and never reaches an API key. Only the six permissions can be mapped, never Admin, Editor or Viewer. - **On a host without Beacon user management**, the session's `Admin` role claim grants every permission. Other role claims named like a permission's role grant it only with `Beacon:Authorization:Roles:HonourHostRoleClaims: true` (default `false`). - **Default roles are never privileged.** `Beacon:Authentication:Oidc:DefaultRoleName` and `Beacon:Mcp:Callers:Users:DefaultRoleName` may not name the Admin role or a functional role; startup fails otherwise (for the MCP setting, while user callers are enabled with `AutoProvision`). ### Separation of duties With `Beacon:Authorization:SeparationOfDuties:Enabled: true` (off by default), assignments that would give one user both permissions of a conflicting pair are refused with `400`, also when an Admin assigns them: - Approver + SecurityAdmin - Auditor + SecurityAdmin - Auditor + UserAdmin - UserAdmin + Approver - UserAdmin + MigrationOperator With it on, also: an `IdpRoles:Map` that maps onto both permissions of a pair fails startup; at sign-in, a mapped permission that would conflict with the user's stored functional roles is not added; a caller who holds both permissions of a pair anyway (roles assigned before the switch) holds **neither**, with a warning at most once an hour; and the startup log counts the active users holding a conflicting pair. Admins hold every permission and are not separated by these pairs; with the approval workflow on, an Admin still cannot approve a change they requested. ### Sessions and the realtime hub With user management on, a `Beacon.Auth` session's role claims are replaced by the stored roles at every revalidation (`Beacon:Authentication:Session:RevalidateMinutes`, default 5), and a role change renews the user's security stamp, so their sessions end. See [User management → Sign-in sessions](https://moberghr.github.io/beacon/security/user-management/#sign-in-sessions). The realtime hub (`/beacon/api/hub`) admits a signed-in caller with read permission, or one holding **Approver** or **MigrationOperator** (their pages refresh from hub events). Other functional-only users get `403` at negotiation and `GET /beacon/api/auth/me` reports `realtimeEnabled: false`. API keys get no hub connection. ## MCP safety locks Read-only enforcement and PII detection are MCP settings, globally and per project, but by default the deployment pins both on: `Beacon:Mcp:ForceReadOnly` and `Beacon:Mcp:ForcePiiDetection` default to `true`. Two more locks are off by default: `ForceNoContentRetention` (questions, SQL and free-text errors never persisted) and `ForbidRowDataToLlm` (no value read from a data source reaches the LLM provider). `Beacon:Mcp:Audit:RequireContent` (off by default) pins content retention **on**, so the MCP audit keeps question and SQL text; it cannot be combined with `ForceNoContentRetention` (the host does not start with both on), see [Keeping audit content](https://moberghr.github.io/beacon/mcp/observability-and-audit/#keeping-audit-content-the-requirecontent-lock). While a lock is on, nobody can change the setting it pins, an Admin included (`409`, `/errors/setting-locked`); `GET /beacon/api/mcp/settings/locks` lists the pinned settings, and the MCP settings page shows them disabled. Stored values a lock overrides are kept, have no effect, and are listed in one startup warning. Only host configuration lifts a lock; see [Configuration → Deployment locks](https://moberghr.github.io/beacon/getting-started/configuration/#deployment-locks). ## Host seams All defaults are registered with `TryAdd`: register your own implementation before `AddBeaconServices`, or replace the registration. | Seam | What it decides | |---|---| | `IBeaconAuthorizationProvider` | global read and write permission (`HasReadPermissionAsync`, `HasWritePermissionAsync`), and a veto on functional permissions (`AllowsFunctionalPermissionAsync`) | | `IProjectMembershipSource` | where memberships come from (for example identity-provider groups). `IsAvailable` is read once at startup; `GetMembershipsAsync` returns the caller's role per project. A `null` result or an exception denies every restricted project; unknown roles are dropped. With a custom source Beacon's membership routes answer `409` | | `IRunAsCeilingSource` | the global role that caps an actor who is not the request's caller (default: the user's stored roles) | | `ISystemWorkScope` | marks a DI scope as Beacon's own work. A host that runs Beacon's jobs on its own job host **must** call `MarkAsSystemWork()` on each job's scope; an unmarked scope without a request gets no project access | | `IRunAsScope` | runs a scope as the owner of scheduled work; set by Beacon's job entry points, not by hosts | | `IExecutionAccessCheck` | the execute-time re-check of the scope's caller on a set of data sources, right before SQL or LLM work | | `IBeaconPermissionAccessor` | the caller's `BeaconPermissionSet` (`IsAdmin`, `Permissions`, `Holds(permission)`), read from stored roles once per request | | `IProjectAccessEvaluator` | the whole project decision, returning a `ProjectAccessScope` (`IsUnrestricted`, `For(action)`, `RoleIn(projectId)`, `Allows(projectId, action)`) | | `IBeaconActorAccessor` | the caller as a resource owner (`BeaconActor`: `UserId`, `IsAdmin`); an API key is its owner | ### Custom authorization provider ```csharp using Beacon.Core.Authorization; public sealed class MyAuthorizationProvider(IBeaconUserContext userContext, IMyPermissionService permissions) : IBeaconAuthorizationProvider { public Task HasReadPermissionAsync(CancellationToken cancellationToken = default) => permissions.HasPermissionAsync(userContext.UserId, "beacon.read"); public Task HasWritePermissionAsync(CancellationToken cancellationToken = default) => permissions.HasPermissionAsync(userContext.UserId, "beacon.write"); } // in AddBeaconServices(..., options => { ... }) options.Authorization.Enabled = true; options.AddAuthorizationProvider(); ``` The provider is the global ceiling; which projects and resources a caller reaches is decided by project access. The resource-level members (`AuthorizeAsync`, `AuthorizeNewResourceAsync`, `GetAccessibleResourceIdsAsync`) and `AuthorizationOptions.EnableResourceLevelAuthorization` are `[Obsolete]`, never called, and removed in 5.0. ## Configuration Code-only, in the `AddBeaconServices` lambda: | Option | Default | What it does | |---|---|---| | `Authorization.Enabled` | `false` | Enforces the global role on `/beacon/api`. With user management on, stored roles cap callers either way | | `Authorization.ProviderType` (`AddAuthorizationProvider()`) | `DefaultAuthorizationProvider` (allows all); `DatabaseAuthorizationProvider` with user management | The global-permission provider | Bound from `Beacon:Authorization` and validated at startup: `RequireProjectMembership`, `Roles:*`, `Roles:HonourHostRoleClaims`, `IdpRoles:*` and `SeparationOfDuties:Enabled`. See [Configuration → Authorization](https://moberghr.github.io/beacon/getting-started/configuration/#authorization). ## Audit and security events Security-relevant actions (sign-ins, users and roles, API keys, project access, data sources, delivery, settings, approvals, ownership, alert tasks, curation, data contracts and migration jobs) are recorded in the append-only, signed [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/), which an Auditor reads and verifies. Beacon also keeps `McpAuditLog` (every MCP tool call and the statements it ran, read by Auditors) and `AppSettingHistory` (admin settings, MCP settings and migration-write-target changes), and writes security-relevant actions as structured log events 7101–7123 with identifiers only; those log events are not stored by Beacon, so route them through your log pipeline. See [MCP observability and audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/) and [Security model](https://moberghr.github.io/beacon/security/). Queries, subscriptions, recipients, data contracts and migration jobs also record who created them and who changed them last. They implement `IAttributedEntity` (`Beacon.Core.Data.Entities.Base`: `CreatedByUserId` and `ModifiedByUserId`, the `Users.ExternalId` of the creator and of the last to change it, and `ModifiedTime`). Beacon sets them from the request's caller (`BeaconActor.UserId`, with `Attribution.RecordChangeBy`), never from a request body; they are null for resources created or changed before they were recorded. An Auditor or a UserAdmin lists a user's resources with `GET /beacon/api/users/{id}/owned-resources` (see [User management: Owned resources](https://moberghr.github.io/beacon/security/user-management/#owned-resources)). ## Troubleshooting | Symptom | Check | |---|---| | Everyone can do everything | `options.Authorization.Enabled = true`, or user management on; `app.UseAuthorization()` in the pipeline | | `403` on every request | the provider returns `true` for authenticated users; the claims transformation runs and adds `ClaimTypes.Role` | | A user sees "not found" for a project that exists | the project is restricted and the user is not a member; `404` is the answer for anything the caller may not read | | A user with a functional role sees no projects | functional roles grant no project data; add a Viewer or Editor role | | An Admin is refused after a role change | permissions and the Admin bypass follow the stored roles now, not the session's claims | | Scheduled runs show "Skipped: owner not authorized." | the owner lost access; restore it or have an Admin or UserAdmin assign another owner | | A background job reaches nothing | the job's DI scope was not marked with `ISystemWorkScope.MarkAsSystemWork()` | ## Security practices 1. Enforce on the server; client-side checks are for display only. 2. Use HTTPS and keep the session cookie's flags as Beacon sets them. 3. Keep roles in Beacon's user store or your identity provider, not hard-coded in a claims transformation. 4. Behind a reverse proxy, configure forwarded headers so the per-IP login and setup throttles see the client, not the proxy. 5. Alert on `AccessRefused` (7108) spikes and on the UserAdmin events above. 6. Keep `RevalidateMinutes` short and `AbsoluteLifetimeHours` to a working day; use **Sign out everywhere** when an account may be compromised. 7. With single sign-on, turn local password sign-in off and keep a break-glass account ([Local password sign-in](https://moberghr.github.io/beacon/security/user-management/#local-password-sign-in)). 8. Restrict sensitive projects (new ones already are), give each data source a login that sees only what its projects should, and configure its database in the connection. 9. Turn on separation of duties where one person must not hold both sides of a pair. ## Upgrading from earlier versions ### Upgrading from 4.5: project access Projects became the authorization boundary on the REST API, the UI, the MCP playground, MCP tool calls and scheduled work. - **Schema.** Apply the `AddProjectAuthorization` and `AddContractOwnerNotAuthorized` migrations (both providers) as you apply the others; Beacon does not migrate its schema at startup. Existing projects stay open and existing data sources stay reachable (`open_when_unlinked = true`). - **New behaviour without configuration.** New projects are restricted with the creator as project admin; new data sources are Admin-only until linked; sessions without a stored user reach nothing while user management is on; a refusal answers `404`/`403` through Beacon's own exception mapping. - **API keys.** A key's project list applies on every REST route, not only MCP; new keys must name at least one project. Keys **without** a list now reach their owner's projects on MCP (before: none). Re-issue them with a list. - **MCP.** `ask`, `query`, `dry_run` and `feedback` need Query (Editor) in the project; mapped Entra user callers need membership of restricted projects. - **Scheduled work runs as its owner** and is skipped when the owner may no longer do it. Assign owners (`PUT …/owner`, Admin or UserAdmin) to ownerless subscriptions, contracts and actors before restricting projects. - **Background jobs on a host's own job runner** must mark each job scope with `ISystemWorkScope.MarkAsSystemWork()`. - **Every query execution goes through the execution gate**, with `Beacon:Query:MaxExecutionRows` and `StatementTimeoutSeconds`; check stored queries and subscriptions after upgrading (comments no longer swallow clauses, delivered rows are PII-masked, a statement's own bound above the cap is lowered to it, SQL naming another database is refused). - **MCP read-only enforcement and PII detection are locked on** by default; **data migration is off** unless `Beacon:Migrations:Enabled` is `true`. - **Recommended steps for a regulated deployment:** give each data source a login per boundary and configure its database; add members and restrict sensitive projects; re-issue integration keys with project lists; review shared data sources on each project's **Access** card; then set `Beacon:Authorization:RequireProjectMembership: true`. - **Public API.** New project-access types in `Beacon.Core.Authorization` (`IProjectAccessEvaluator`, `ProjectAction`, `ProjectRole`, `IProjectMembershipSource`, `ISystemWorkScope`, `IRunAsScope`, `IExecutionAccessCheck`, …); several positional records gained members and need a recompile. ### Upgrading: functional permissions Existing Admins are unchanged: they hold every permission, and the new roles are optional. - The six roles are created as level-0 system roles when roles are next listed or seeded; a role of yours with one of those names grants nothing until renamed (or map the permission to another name). - The listed routes carry `BeaconApi.{Permission}` policies and re-check the permission live; a demoted Admin's session is refused on them. - On a host without Beacon user management, role claims named like a permission no longer grant it unless `Roles:HonourHostRoleClaims` is `true`. - `DefaultRoleName` (OIDC and MCP callers) may no longer name the Admin role or a functional role. - UserAdmins create external users only, and no longer open restricted projects or touch protected users; SecurityAdmins no longer turn read-only enforcement or PII detection off or change the base URL. ### Upgrading: sign-in sessions - Every session is checked against the user store at its next request, then every `RevalidateMinutes`; sessions end `AbsoluteLifetimeHours` (default 12) after sign-in, whatever "remember me" asked. - The `AddUserSecurityStamp` migration adds `users.security_stamp` and `must_change_password`. Password changes, disabling, role changes and **sign out everywhere** end the user's other sessions. - `AddBeaconCookieAuthentication` no longer configures Data Protection: call `AddBeaconDataProtection(configuration, environment)` or configure your own; outside Development set `Beacon:DataProtection:ApplicationName` and encrypt the key ring. Everyone signs in again once. - `Beacon:EncryptionKey` is checked in every environment; list a replaced key in `Beacon:PreviousEncryptionKeys`. - See [User management → Upgrading from 4.5](https://moberghr.github.io/beacon/security/user-management/#upgrading-from-45) for sign-in changes (local login, break-glass, forced password change, `RequireMfa`). ### Upgrading: data minimisation - Host data sources are masked by output position; database errors are redacted by class before they reach prompts, signals, callers or the audit; sample values and value grounding are off for a new installation. - Banking identifiers (`iban`, `oib`, `jmbg`, account numbers) match the built-in PII pattern. - Every LLM request carrying row data logs `RowDataSentToLlm` (7123); `Beacon:Mcp:ForbidRowDataToLlm` locks row data out of prompts. - The `DefaultSampleValueCollectionOff` migration changes only a column default. ### Upgrading: change governance The approval workflow became a four-eyes control bound to reviewed content, and was extended to migration jobs, Custom SQL rules and recipient destinations. It stays opt-in. Pending requests made before the upgrade can only be rejected and resubmitted. Apply the `AddQueryChangeIntegrity` and `AddChangeApprovalRequests` migrations. See [Approvals](https://moberghr.github.io/beacon/security/approvals/). ### Upgrading: grounding integrity Feedback no longer creates golden examples (an Admin promotes candidates), curation needs the project admin role, every curation binds to the content reviewed, and grounding SQL is checked before it is stored. Apply the `AddGroundingAttribution` and `AddGroundingProvenance` migrations, review existing golden cases that record no promoting user, and add a project admin to each project whose grounding someone other than an Admin should curate. See [Knowledge base](https://moberghr.github.io/beacon/mcp/knowledge-base/#who-curates-grounding). ### Upgrading: security audit trail Security-relevant actions are recorded in an append-only, signed trail an Auditor reads and verifies, governed resources record who created and last changed them, and the MCP audit keeps the statements each call ran. Apply the `AddSecurityAuditTrail` migration (additive; nothing is backfilled). See [Security audit trail: Upgrading](https://moberghr.github.io/beacon/security/audit-trail/#upgrading) for the schema, public API and REST changes. - **Events and attribution start at the upgrade.** Existing resources record no creator or last modifier until they are next changed. - **Set `Beacon:Audit:IntegrityKey`** before you change the encryption key or the caller hash key; otherwise events signed under the old derived key verify as *unknown key* once that secret leaves the configuration. - **Schedule the verify and purge jobs** (`ISecurityAuditVerificationService.VerifyNewEventsAsync`, hourly; `ISecurityAuditRetentionService.PurgeExpiredAsync`, daily); the sample host schedules both. Events are kept forever until `Beacon:Audit:RetentionDays` (365 to 36,500) is set. - **Alert on** `beacon.security_audit.write_failures` and `beacon.security_audit.verification_failures` above zero, and ship the `Beacon.Audit` events 9104 to 9114 to your SIEM. - **Reads of audit data fail closed**: the security audit, MCP audit and owned-resources routes record each read first and fail when that record cannot be written. - **Owner changes** of subscriptions, data contracts and AI actors answer `409` when the owner changed since it was read. - **Where content is retained, the MCP audit now stores the SQL of every statement a call ran.** Review this against your privacy policy; `Beacon:Mcp:Audit:RequireContent` pins it on, `ForceNoContentRetention` pins it off. - **A host that wires sign-in without `AddBeaconServices`** must register `ISecurityAuditRecorder`; `TaskService` and `AppSettingsService` now take one. ### Upgrading: system catalog SQL that reads the engine's catalog, statistics or query history (`information_schema`, `pg_catalog`, `pg_stats`, `sys`, `sysobjects`, Snowflake `QUERY_HISTORY` and `RESULT_SCAN`, BigQuery `INFORMATION_SCHEMA`, …) is now refused on every data source with the gate code `system_catalog`, whatever the read-only setting. Stored queries, subscriptions, Custom SQL rules and saved-query tools that read such a relation fail from the upgrade: rewrite them against the source's own tables. See [Database names in SQL](#database-names-in-sql). ## See also - [User management](https://moberghr.github.io/beacon/security/user-management/): accounts, sign-in, sessions, functional roles on the Users page - [API keys](https://moberghr.github.io/beacon/security/api-keys/): scopes, project lists, expiry - [Approvals](https://moberghr.github.io/beacon/security/approvals/): the four-eyes workflow - [Security model](https://moberghr.github.io/beacon/security/): the controls end to end - [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/): every key --- # Approvals > Four-eyes change approval: what it reviews (query changes, migration jobs, Custom SQL rules, notification destinations), how a decision is bound to exactly what the reviewer saw, who may approve, strict mode, and the API. Source: https://moberghr.github.io/beacon/security/approvals/ The approval workflow makes a second person review every change that decides **what SQL runs** or **where data goes** before it takes effect. It is **off by default**. A host cannot tell a new installation from an existing one, so turning it on is the host's decision, and Beacon logs a startup note while it is off. ```csharp builder.Services.AddBeaconServices(builder.Configuration, options => { options.EnableApprovalWorkflow(x => { x.RequireApprovedVersionToRun = true; // strict mode, recommended for regulated deployments }); }); ``` | Option | Default | What it does | |---|---|---| | `Enabled` (set by `EnableApprovalWorkflow`) | `false` | Turns the workflow on. | | `AllowSelfApproval` | `false` | Let the requester (and others who shaped a change) approve it themselves. Only for a deployment with a single possible approver; Beacon warns at startup while it is on, and warns when fewer than two users can approve while it is off. | | `RequireApprovedVersionToRun` | `false` | Strict mode: a subscription (and a saved-query MCP tool) runs only a version that carries an approval for exactly its content. | | `CoversMigrationJobs` | `true` | New and changed migration job definitions need approval before any run. | | `CoversCustomSqlRules` | `true` | New and changed Custom SQL rules of data contracts need approval before any evaluation. | | `CoversRecipientDestinations` | `true` | New recipients and changed destinations (type, address, headers) receive nothing until approved. | | `CoversDataSourceConnections` | `false` | New data sources, and connections changed in anything but their secrets, connect to nothing until approved. | | `CoversLlmConnection` | `false` | A change of the stored LLM connection's provider, endpoint, region or authentication mode is staged until a live Admin approves it. | | `CoversAdminRoleGrants` | `false` | Giving a user the Admin role waits until another live Admin approves it, whatever `AllowSelfApproval` says. | | `LlmConnectionRequestDays` | `7` | Days a staged LLM connection change waits for a decision before it expires (held between 1 and 30). | The per-kind switches apply only while the workflow is enabled. Configuration is in code; there are no `appsettings` keys for it. ## Who approves Decisions need the **Approver** [functional permission](https://moberghr.github.io/beacon/security/authorization/). Admins hold it too. - The **requester never approves their own change**, however either of them signed in, and neither does anyone else who shaped it (for an AI actor's change: the plan's reviewer and whoever asked for its revisions). Admins are no exception. - Four eyes binds per Beacon user. Do not give one person two accounts that can approve. - API keys hold no functional permission and MCP has no approval tool, so neither can approve anything. - Approvers see every request on **Approvals** (`/beacon/approvals`), including requests for queries they otherwise cannot read; other users see the requests of resources they can read. A change of the LLM connection and a grant of the Admin role are the exception: only a live Admin reads and decides them, their requester included. - A decision on a change request (approving or rejecting) checks the reviewer again in the user store when it is made, after the lock on what it decides: a reviewer disabled, archived, signed out everywhere or stripped of the Admin role or the Approver permission while the decision waited is refused (`403`) and nothing is decided. A role change, a disable or an archive of the reviewer that starts meanwhile waits until the decision commits. ## Query changes **What counts as a change:** the query's executable content: its steps (added, removed, reordered), each step's name, data source, SQL and parameters, and the final query. Every character counts, whitespace and line endings included. A change to only the query's name or description goes live directly. **Submitting.** Saving a change files a *PendingApproval* version with exactly the submitted content and its SHA-256 content hash, and an approval request naming the requester and the active version the change was made on. The live query is not touched. - A query has one pending request. A new submission by the same person supersedes their earlier request; while **another** person's request waits, every submission of the query (a save, a restore, an AI actor's refinement) is refused with `409` until a reviewer decides. - A **new query** goes live only once approved; until then it has no steps and runs nothing. - A **version restore** is submitted like a save. - Saving a query whose stored SQL differs from its active version (changed outside a versioned save) brings that SQL under review. Whoever saves it becomes its requester, so another Approver should review it. **Reviewing.** The request detail shows the proposed version, the active version and a step-by-step diff (SQL, data source and parameters per step), the proposal's content hash, and whether the caller may approve and why not. **Deciding.** Approving sends back the content hash the reviewer saw (`expectedContentHash`, required). The approval applies only if: - the request is still pending (a second decision answers `409` and never overwrites the first); - no other version went live since the change was made (otherwise it is stale; submit again); - the proposal still holds the submitted content, and that content is what the reviewer was shown; - every step passes the save checks again (the execution gate and the data source's metadata). The decision, the move of the query to the new version and the version statuses commit in one transaction. Rejecting marks the proposal Rejected; a live version is never marked Rejected. ## Runs execute reviewed SQL With the workflow on, runs that happen without the person who wrote the SQL (scheduled subscription runs, test deliveries, subscriptions an AI actor creates, and [saved-query MCP tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/)) execute only reviewed content: - A subscription run executes the stored steps only when they are the content of the query's active version. Otherwise the run is recorded as **Skipped** ("query change not approved"), the subscription and its Control Tower row show **Not approved**, and a test delivery answers `409`. - A query stored before the workflow was on and never versioned since runs as stored outside strict mode, logging a warning each time; Beacon counts such queries at startup. Save the query to submit it. - In **strict mode** a run also needs an approval for exactly the active version's content hash, so versions that went live while the workflow was off, or were approved before Beacon recorded hashes, do not run until approved. Interactive previews run the caller's own SQL under the caller's permissions and are not compared. ## AI actors With the workflow on, an actor's new query or refinement is validated without running it (the save checks plus the engine's non-executing validation: `EXPLAIN` without `ANALYZE` on PostgreSQL, MySQL and Snowflake, `sp_describe_first_result_set` on SQL Server and Synapse) and then submitted with the actor's creator as the requester. The creator cannot approve it. See [AI actors](https://moberghr.github.io/beacon/features/ai-actors/). ## Migration jobs, Custom SQL rules and recipients These are reviewed as **change requests** with the same rules: four eyes, one decision while pending, and an approval bound to what the reviewer was shown (`expectedPayloadHash`) and to the definition the resource still holds. | Kind | Reviewed | Until approved | |---|---|---| | Migration job | New job, or a changed source, steps, destination, mode or transformation | Every run, manual or scheduled, is refused and recorded as failed. | | Custom SQL rule | New rule, or changed SQL or data source | The rule is left out of every evaluation. | | Recipient destination | New recipient, or changed type, destination or headers | Deliveries skip the recipient; each skip is logged. Attaching an existing recipient to a subscription is not reviewed. | | Data-source connection (`CoversDataSourceConnections`) | New data source, or any change of its connection but the secrets (host, port, database, login, any other setting) | The source stays listed but connects to nothing: every query, run and metadata scan of it is refused. A new source has no catalog until approved. | | LLM connection (`CoversLlmConnection`) | A change of the provider, endpoint, region or Bedrock authentication mode in [Admin settings](https://moberghr.github.io/beacon/security/admin-settings/), or a first stored provider over the host's `Beacon:LLM` | Nothing changes until approved: the stored connection and models stay in use. The change is staged, encrypted. | | Admin role grant (`CoversAdminRoleGrants`) | A user created with the Admin role, or the Admin role added to a user's roles | The user does not hold the Admin role. The rest of the save (other roles, the new user) applies at once. | When a change lies only in a part the review shows masked (a token, a header value), the approval must carry a comment saying how it was checked. A data-source connection is reviewed by its host, port, database and login and the names of the other settings that changed, never a secret, with the projects the source is linked to now. A change of the secrets alone (a rotated password) applies at once; a connector that cannot read the connection setting by setting (BigQuery, Databricks, CloudWatch) shows only a fingerprint, so every change of such a source needs approval with a comment. Approving checks the server against the declared engine first, once the source still holds the connection the request proposes: a mismatch, a server that cannot be reached or whose engine cannot be read, and a disabled source (enable it first) answer `409` and the request stays pending. A source created under the workflow and never approved shows that it waits for its first approval to everyone who reads it, also when no request waits any more (its first request was rejected): save its connection again to submit it. A change that waits for approval needs a requester Beacon can name (`403` otherwise); a rename, a metadata option or a rotation of the secrets does not. After the approval the source's metadata is refreshed (an API source created while held imports its endpoints). Test connection stays outside the approval. A requester who is no Approver withdraws a held change by deleting the source or saving the approved connection back. An LLM connection change stages the whole resulting connection: the four governed fields, the models from the same save and every credential (a new one, or the stored one), encrypted with `Beacon:EncryptionKey`. The rest of the save (the base URL, the budget, the limits) applies at once. A change of credentials or models alone is not reviewed and applies at once, and so does testing the connection. Only a live Admin reads and decides it, the requester included, and never the requester while self-approval is off, so it needs a second Admin. The reviewer sees the provider, region, authentication mode and models, the endpoint as scheme and host with a fingerprint of the whole endpoint, a fingerprint of each credential and of the connection it was made from, and never a credential, the endpoint's path or the staged values; a change that lies only in the endpoint's path or query is approved with a comment. One change waits at a time: another person's save of the connection answers `409`, the requester's own earlier change is superseded. An approval applies only while the stored connection is still the one the change was made from (a key rotated since makes it stale: `409`, and the change is superseded), and only while its provider is installed. A save is judged against the stored connection, never a copy another Beacon process may still cache. Saves of the connection and decisions on its changes take one lock in Beacon's database (a transaction-scoped advisory lock on PostgreSQL, `sp_getapplock` on SQL Server) before they read it, so they run one after the other: a key rotated while an approval is being decided either lands first and makes the change stale, or waits and is judged against the connection approved. The approval writes the connection as the requester's change in the settings history, and applies it at once; when this process cannot apply it, the approval stands and answers `409` saying the previous connection stays in use here until Beacon restarts. What a change staged is wiped as soon as it can no longer be applied: approved, rejected, withdrawn, superseded, stale, or expired after `LlmConnectionRequestDays` (an expired change answers `409`). The re-encryption run moves the staged values to the current key; a waiting change made under a previous key keeps the previous key needed until it is decided or expires. A grant of the Admin role is recorded by ids only: the user's `Users.Id` and the role's id and name, never a user name, e-mail address or external id; the reviewer sees the user by display name (or "User {id}"), read live. Removing the Admin role, and granting or removing a functional role, are not reviewed and apply at once. Only a live Admin reads and decides a grant, never its requester, also while `AllowSelfApproval` is on, so it needs a second Admin (startup warns with fewer than two); a caller Beacon cannot name cannot request one (`403`). A grant is bound to the user's security stamp as it was when the grant was submitted (in its definition hash, never stored in clear): changing the user's user name, e-mail address or enabled state, or resetting their password, supersedes the pending grant in the same save and tells its requester; any other change that ends the user's sessions (a role change, signing out everywhere, the user's own new password) leaves the grant pending but no longer approvable (`409`), and the grant is requested again. A display-name change keeps it approvable. Saving the same grant again while one waits adds no second request; saving it after the user's stamp changed supersedes the outdated request. An approval applies only while the user still exists, is not archived, does not hold the Admin role yet and holds the stamp the grant was submitted with (`409` otherwise); it adds the role, recorded as assigned by the requester, ends every session the user holds, and records `identity.roles_changed` with `approvals.change_approved`. Under the switch a user provisioned at a first sign-in (SSO, MCP) is never given the Admin role: a host whose default role is Admin provisions the user with no role and logs a Warning (Beacon's own SSO and MCP defaults already refuse Admin at startup). `IUserManagementService.AssignRoleAsync` refuses the Admin role under the switch; first-run setup still makes the super admin an Admin. Identity-provider app roles map onto functional permissions only, never the Admin role. Resources stored before Beacon reviewed their kind, or last saved while the switch was off, keep working as before. ## Publishing a query as an MCP tool While the workflow is on, whoever requested or approved the version a saved-query tool runs cannot be the one who enables the tool (unless `AllowSelfApproval`). See [Saved query tools](https://moberghr.github.io/beacon/mcp/saved-query-tools/). ## API | Method and route | Who | What | |---|---|---| | `GET /beacon/api/approvals/pending` | Approver, or readers of the query | Pending query approvals (paged). | | `GET /beacon/api/approvals/{id}` | Approver, or readers of the query | Request detail: proposal, active version, diff, `contentHash`, `mayApprove`. | | `POST /beacon/api/approvals/{id}/approve` | Approver | Body `{ "comment": "…", "expectedContentHash": "…" }`. | | `POST /beacon/api/approvals/{id}/reject` | Approver | Body `{ "comment": "…" }`. | | `GET /beacon/api/approvals/changes` | Approver, or readers of the resource | Change requests: migration jobs, Custom SQL rules, recipients, data-source connections; the LLM connection's and the Admin role's grants for a live Admin only. | | `GET /beacon/api/approvals/changes/{id}` | Approver, or readers of the resource | Change request detail with its payload hash (`404` for an LLM connection change or an Admin role grant unless the caller is a live Admin). | | `POST /beacon/api/approvals/changes/{id}/approve` | Approver (a live Admin for an LLM connection change, `403` otherwise) | Body `{ "comment": "…", "expectedPayloadHash": "…" }`. | | `POST /beacon/api/approvals/changes/{id}/reject` | Approver (a live Admin for an LLM connection change, `403` otherwise) | Body `{ "comment": "…" }`. | The reviewer is always the signed-in caller, never an id in the request. Decisions push an `ApprovalUpdated` event to the requester and reviewer over the realtime hub. Every submission and decision is recorded in the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/#event-catalogue): `approvals.query_change_*` for query versions, `approvals.change_*` for change requests (with the kind, the resource id and the requests a submission supersedes) and `approvals.ai_plan_*` for AI actor plans. The reviewer is the event's actor, and a decision commits together with its event; a decision on a query change or a change request also names its requester by user id. Neither the proposed content nor a comment is stored in the event. ## Turning it on in an existing installation 1. Make sure at least two people hold Approver. 2. Enable the workflow; leave strict mode off for the first deployment. 3. Check the startup log: Beacon counts queries whose stored SQL is not versioned or approved. 4. Have owners save those queries (submitting them), approve them, then turn on `RequireApprovedVersionToRun`. --- # User Management > Built-in users and roles: internal password and external identity-provider users, first-run setup, functional roles, sessions, local and single sign-on, and the sign-in routes. Source: https://moberghr.github.io/beacon/security/user-management/ Beacon has a built-in user store: internal users who sign in with a password stored in Beacon, external users who sign in through your identity provider, global roles, the six functional roles, and server-checked sessions. It is **off by default**. Who may do what once a user exists is described on [Authorization](https://moberghr.github.io/beacon/security/authorization/). ## Overview - **Internal users** sign in with a user name (or e-mail address) and a password Beacon stores as a PBKDF2-SHA256 hash (600,000 iterations, 32-byte salt; see [Password hashes](#password-hashes)). - **External users** sign in through single sign-on (OIDC), a REST bearer token, or a login API that returns a JWT. Their roles always come from Beacon, never from the token. - **Global roles** Admin, Editor and Viewer; **functional roles** Approver, Auditor, SecurityAdmin, UserAdmin, DataSourceAdmin and MigrationOperator, one administrative duty each, without project data. - **First-run setup** creates the super admin with a setup token. - **Sessions** are checked against the user store on the server and end after an absolute lifetime. - **Soft delete**: archiving a user keeps their history. ## Enabling user management User management is configured in code, in the `AddBeaconServices` lambda: ```csharp builder.Services.AddBeaconServices(builder.Configuration, options => { options.AddBeaconScheduler(); options.Authorization.Enabled = true; options.Authentication.EnableLoginForm = true; options.AddAuthenticationProvider(); options.EnableUserManagement(x => { x.AllowInternalUsers = true; x.MinimumPasswordLength = 12; }); }) .UsePostgreSql(); // The SPA is served at /beacon: the login page is /beacon/login builder.Services.AddBeaconCookieAuthentication("/beacon"); // Data Protection keys for the session cookie (or configure your own). Outside Development set // Beacon:DataProtection:ApplicationName and chain ProtectKeysWith*, or the host does not start. builder.Services.AddBeaconDataProtection(builder.Configuration, builder.Environment); ``` The host also maps the login and setup endpoints (`app.MapLoginEndpoints("/beacon", beaconConfiguration)`, `app.MapSetupEndpoints("/beacon")`) and the login-form and first-run middleware; see [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/) and [Sign-in & identity](https://moberghr.github.io/beacon/integrate/authentication/). | Option (`options.UserManagement.*`) | Default | What it does | |---|---|---| | `Enabled` | `false` | Turns on the user store, the Users page, first-run setup and `DatabaseAuthorizationProvider` | | `AllowInternalUsers` | `true` | Allows password users | | `HostOwnsSignIn` | `false` | The host signs people in, so there is no first-run setup (see [When the host owns sign-in](#when-the-host-owns-sign-in)); code-only | | `MinimumPasswordLength` | `8` | Shortest password accepted | | `RequirePasswordComplexity` | `true` | Requires an upper-case letter, a lower-case letter, a digit and a character that is neither | | `SetupToken` | unset | The first-run setup token; also bound from `Beacon:UserManagement:SetupToken` (see [First-run setup](#first-run-setup)) | | `LocalLogin` | on | Password sign-in; also bound from `Beacon:UserManagement:LocalLogin` (see [Local password sign-in](#local-password-sign-in)) | ## First-run setup While no super admin has ever existed in the database (an archived super admin counts), every page redirects to the setup page, `/beacon/setup`: 1. Open Beacon. You are redirected to **setup**. 2. Enter the **setup token**: the value of `Beacon:UserManagement:SetupToken`, or, when none is configured, the token Beacon prints once to the **process console** (standard error) at startup. The generated token is never written to the log; a log Warning only says that it was printed. 3. Create the super admin (user name, e-mail address, password). The system roles and the functional roles are seeded. 4. Sign in with the new account. A configured `SetupToken` must be at least 32 characters, or the host does not start. Each replica generates its own token, so a deployment with several replicas must configure one. The token is compared in constant time; creation is atomic (a request that loses a race gets `409` and can retry). A wrong or missing token answers `403`. The setup endpoints are anonymous and throttled per client address, separately from the login form. Until the super admin exists Beacon provisions **no other user**: an SSO sign-in or MCP caller that would create a user is refused. `GET /beacon/api/setup/roles` is anonymous only while setup is open. ### When the host owns sign-in A host that embeds Beacon behind its own identity, and decides itself whom Beacon provisions, sets `HostOwnsSignIn`. Then there is no first-run setup: `GET /beacon/api/setup/status` reports setup as completed, no token is printed, `POST /beacon/api/setup/superadmin` is refused, and external users (the host's own provisioning, SSO, MCP callers) are created without a super admin existing. Nobody is ever a super admin; the host grants the Admin role to whoever administers Beacon. ```csharp options.EnableUserManagement(x => { x.AllowInternalUsers = false; x.HostOwnsSignIn = true; }); ``` It is set in code only, never from configuration: a configuration value that turned it on would let the first caller of a fresh installation be provisioned without setup. ## Managing users An Admin or a UserAdmin manages users on **Users** (`/beacon/users`): - **Add user** creates an internal or an external user with roles. Internal users are an Admin's to create; a UserAdmin creates external users only. A password an Admin sets must be replaced at first sign-in (see [Passwords set by an administrator](#passwords-set-by-an-administrator)). - **Edit user** changes the display name, enabled state and roles. The user name and e-mail address are an Admin's to change. - **Sign out everywhere** ends every session of the user. - **User names and e-mail addresses** are trimmed. A user name needs 3 to 100 characters and no control characters; an e-mail address is optional, but when given it must be one plain address (no display name, no list) of at most 255 characters. Otherwise creating the user answers `400`. Editing a user keeps a user name or e-mail address sent back exactly as stored, surrounding whitespace included, and trims and checks only the fields that change, so an account stored before these rules, or provisioned at an identity-provider sign-in, can still be saved or disabled. First-run setup applies the same rules and refuses a password confirmation that differs. Users provisioned at an identity-provider sign-in keep the provider's values. - Disabling or archiving a user revokes their [API keys](https://moberghr.github.io/beacon/security/api-keys/#when-the-owner-leaves); re-enabling revokes any key still active instead of restoring one. Set a **display name** for everyone who should be added to projects: project member search matches display names only, and other users see people by display name (or "User {id}"), never by user name or e-mail address. ## Roles | Role | Level | Read | Write (create, edit, run ad hoc) | |---|---|---|---| | **Admin** | 3 | Yes | Yes | | **Editor** | 2 | Yes | Yes | | **Viewer** | 1 | Yes | No | | Functional roles | 0 | No | No | A **super admin** (`IsSuperAdmin`, the first-run account) passes every check. The global role is a ceiling, not a grant: [project access](https://moberghr.github.io/beacon/security/authorization/#project-access) decides where a user may use it. Deleting and archiving are the project action **Manage**, which a project admin holds, an Editor holds in an open project, and the recorded owner of a resource holds. A Viewer reads and runs saved SQL only in projects they may use. Some resources belong to a person: an alert task is worked by its assignee, a data contract is changed by its owner, an AI actor by its creator. Only an Admin or a UserAdmin gives a subscription, data contract or AI actor a new owner. ### Functional roles Each functional role grants one [functional permission](https://moberghr.github.io/beacon/security/authorization/#functional-permissions) and **no project data**: it is a level-0 role with neither read nor write. A user who should also use projects needs a Viewer or Editor role beside it. Admins hold every permission. | Role | The duty | |---|---| | **Approver** | Approves and rejects changes under the [approval workflow](https://moberghr.github.io/beacon/security/approvals/); reads every project's approval requests | | **Auditor** | Reads the MCP audit, the MCP and admin settings (secrets masked) and every user's API keys | | **SecurityAdmin** | Changes the MCP and admin settings (not the LLM connection or base URL), recipients, MCP tool publishing and repository tokens | | **UserAdmin** | Manages users (external users only), their roles, sessions and API keys, resource owners and the members of any project; restricts projects | | **DataSourceAdmin** | Creates, tests and deletes data sources; links a data source it can read into a project it administers | | **MigrationOperator** | Runs data migration between data sources it may query and write | **Assigning them.** `GET /beacon/api/users/roles` lists every role, with the `permission` a functional role grants. Roles are assigned when a user is created or with `PUT /beacon/api/users/{id}/roles` (`{ "roleIds": [1, 4] }`, the complete set; `204`, `404` for an unknown user, `400` for an unknown role or a separation-of-duties conflict). While the [approval workflow](https://moberghr.github.io/beacon/security/approvals/#migration-jobs-custom-sql-rules-and-recipients) reviews grants of the Admin role (`CoversAdminRoleGrants`), adding the Admin role, on a new user or an existing one, waits for another live Admin's approval while the rest of the save applies. Role names follow `Beacon:Authorization:Roles` (see [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/#functional-permissions-role-names-identity-provider-app-roles-separation-of-duties)). **On the Users page** the dialogs list roles in two groups, **Global roles** and **Functional roles**. For a UserAdmin who is not an Admin, **Add user** offers external users only, the user name and e-mail address are read-only, the Admin and functional roles are disabled, and the dialog opens read-only for their own account and for protected users (Admins, super admins and functional role holders). **What a UserAdmin cannot do** (`403`): assign or remove the Admin role or a functional role; create an internal user or change a user name or e-mail address; change their own account, roles or project membership; touch a protected user's account, roles, memberships or API keys; open a restricted project; make themselves the owner of scheduled work. Every user created logs `UserCreated` (7114) and every role change `UserRolesChanged` (7113), with ids only. See [Authorization → Rules](https://moberghr.github.io/beacon/security/authorization/#rules) for what a UserAdmin can do and how to watch it. **From the identity provider.** With `Beacon:Authorization:IdpRoles:Enabled`, app roles mapped in `IdpRoles:Map` add their permission to an SSO session at sign-in. They are not stored, count for `IdpRoles:ClaimLifetimeMinutes` (default 60), do not show on the Users page and never reach an API key. **Default roles.** The role SSO and MCP provisioning give new users (`DefaultRoleName`) may not be the Admin role or a functional role; startup fails otherwise. ### Project roles and membership The global role caps what a user may do; [project access](https://moberghr.github.io/beacon/security/authorization/#project-access) decides where. **New projects are created restricted**, with their creator as project admin, when Beacon manages membership; projects that existed before project access stay open until restricted. An open project is used by everyone with a role; a restricted one only by its members and Admins. | Project role | In the project | |---|---| | **Viewer** | reads its resources and runs its saved SQL | | **Editor** | also works ad hoc and creates and changes resources; deletes what they own (in an open project, anything) | | **Project admin** | also deletes any of its resources, manages its members and links data sources | A global Viewer is a Viewer in every project, whatever their membership; Editors and Admins act with their membership role (and in an open project with the higher of that and their global role). Project admins, UserAdmins and Admins manage members on the project page's **Members** tab. An Admin or a UserAdmin restricts a project; only an Admin opens a restricted one. A disabled or archived user keeps their membership rows, which grant nothing. Memberships live in Beacon's database while user management is on. A host can supply its own `IProjectMembershipSource` (for example from identity-provider groups); Beacon's membership routes then answer `409` and the UI hides the Members tab. See [Authorization → Host seams](https://moberghr.github.io/beacon/security/authorization/#host-seams). ## Authentication providers | Provider | Signs in | |---|---| | `DatabaseAuthenticationProvider` | internal users against Beacon's user store | | `JwtExternalApiAuthenticationProvider` | external users through your login API, which returns a JWT | | `HybridAuthenticationProvider` | internal users first, then the external login API | Register one with `options.AddAuthenticationProvider()`. Single sign-on (OIDC) is separate: `builder.Services.AddBeaconOidcAuthentication(builder.Configuration)` with the `Beacon:Authentication:Oidc` section, see [Single sign-on admission](#single-sign-on-admission). ### JwtExternalApiAuthenticationProvider The login form posts the credentials to your login API and validates the token it returns. JWT options are code-only (there is no configuration section): ```csharp builder.Services.AddBeaconJwtAuthentication(jwt => { jwt.ExternalLoginEndpoint = "https://idp.example.com/api/auth/login"; jwt.EnableBearerAuthentication = true; // also accept Authorization: Bearer on the REST API jwt.Validation.JwksEndpoint = "https://idp.example.com/.well-known/jwks.json"; // or Validation.SigningKey jwt.Validation.ValidIssuer = "https://idp.example.com"; jwt.Validation.ValidAudience = "beacon"; jwt.AccessTokenClaim = "scope"; // only for a provider that does not set typ: at+jwt }); ``` With `ExternalLoginEndpoint` or `EnableBearerAuthentication` set, the host does not start without a signing key or JWKS endpoint, at least one issuer and one audience; issuer, audience and lifetime validation must stay on, and `ClockSkew` may be at most five minutes. A `SigningKey` is at least 32 bytes (use base64 of 32 or more random bytes), and `JwksEndpoint` and `ExternalLoginEndpoint` are `https` URLs (`http` to a loopback host is accepted in Development only). With `ExternalLoginEndpoint` set, `AddBeaconJwtAuthentication` makes `HybridAuthenticationProvider` the sign-in provider whenever user management holds internal users: internal users sign in first, and the external login API is the fallback. Without user management, or with `AllowInternalUsers` off, the external login is the provider on its own. Tokens are accepted only with `RS256/384/512`, `PS256/384/512` or `ES256/384/512` from a JWKS endpoint, or `HS256/384/512` with a signing key, and only with the header `typ` `JWT`, `at+jwt` or none at all; any other declared type (a logout or event token, for one) is refused. The token must be an **access token**, shown by positive evidence: a Microsoft Entra token names its client (`azp` or `appid`) and carries `scp` or `roles`; a token from any other issuer has the header `typ: at+jwt` or carries the claim named by `AccessTokenClaim`. A token with `nonce`, `at_hash` or `c_hash` is an ID token and is refused. `AccessTokenClaim` may not name a claim ID tokens carry (`sub`, `aud`, `nonce`, `azp`, `roles`, …); startup fails otherwise. When the SSO authority issued the token, it must also pass the [admission rules](#single-sign-on-admission). A token proves identity only. Sign-in succeeds only for an **existing, enabled, external Beacon user** the token names, and the session carries that user's Beacon roles. Without user management every login-form JWT and REST bearer token is refused. Every failure answers "Invalid username or password." and the log never carries the user name. ### Option 2: External identity provider (JWT/OAuth) External users must exist in Beacon before they sign in with a token (single sign-on provisions them on first sign-in). Pre-register them on the Users page, with `POST /beacon/api/users/external`, or in code: ```csharp var users = serviceProvider.GetRequiredService(); await users.CreateExternalUserAsync(new CreateExternalUserRequest { ExternalId = "subject-from-the-token", // the token's sub (or ClaimsMapping.UserIdClaim) UserName = "john.doe", Email = "john@example.com", DisplayName = "John Doe", RoleIds = [editorRoleId] }); ``` Beacon finds the user by the token subject together with the token issuer: - The subject is `ClaimsMapping.UserIdClaim` (`sub` by default). When a custom claim is configured and missing, the token is refused; Beacon never falls back to `sub`. - A user stored under the token issuer (as SSO stores users) matches. A user pre-registered without an identity provider matches on the subject alone only when exactly one issuer is configured. - Internal users and super admins never match a token. A disabled or archived match refuses the token. - Users created by MCP caller provisioning (keyed on the Entra `oid`) get no REST session from a bearer token. SSO-provisioned users match a bearer token only when its `iss` and `sub` equal the ID token's. For Entra, a v2.0 access token issued to the same application. ### Custom authentication provider ```csharp public sealed class MyAuthenticationProvider(IMyAuthService auth) : IBeaconAuthenticationProvider { public async Task AuthenticateAsync(string username, string password, CancellationToken cancellationToken = default) { var result = await auth.ValidateCredentialsAsync(username, password, cancellationToken); if (!result.Success) { return AuthenticationResult.Failed("Invalid username or password."); } return AuthenticationResult.Succeeded(new AuthenticatedUser { UserId = result.User.Id, UserName = result.User.UserName, Email = result.User.Email, DisplayName = result.User.FullName, Roles = result.User.Roles }); } [Obsolete("Beacon never calls it.")] public Task ValidateSessionAsync(string userId, CancellationToken cancellationToken = default) => Task.FromResult(true); public Task SignOutAsync(CancellationToken cancellationToken = default) => Task.CompletedTask; } ``` While local login is off, whichever provider authenticated a password, only a break-glass account signs in (see below). ## Single sign-on admission With OIDC enabled, Beacon admits only subjects that pass its rules, **before** it provisions anything: the tenant allow-list (`AllowedTenants`, or an explicit `AllowAnyTenant`), the guest block (`BlockGuests`, on by default), then the optional `RequiredRoles`/`RequiredGroups`. A bearer token on the REST API, and a token returned to the login form, that the SSO authority issued passes the same rules. A refused sign-in is logged at Warning with the reason, tenant, issuer and a hash of the subject. With `RequireMfa: true` (off by default) an interactive SSO sign-in must show multi-factor authentication in its ID token (`amr` containing `mfa`, or an `acr` listed in `AcceptedAcrValues`) or it is refused and the login page says so. With Microsoft Entra ID, request `amr` as an optional ID-token claim. `RequireMfa` applies to the interactive SSO sign-in. Bearer tokens (REST and `/beacon/mcp`), API keys, the login-form JWT flow and password sign-in (break-glass included) are authenticated by their own credentials; set the MFA requirement for tokens in the identity provider that issues them. SSO provisions new users **without a role** unless `DefaultRoleName` is set; they see "Your account has no access yet" until a role is assigned. The full option table is on [Configuration → OIDC / SSO](https://moberghr.github.io/beacon/getting-started/configuration/#optional-oidc--sso). ## Local password sign-in Password sign-in through the login form (`POST /beacon/api/auth/login`) is **on** by default. A deployment where people sign in through single sign-on should turn it off, so every sign-in passes the identity provider's MFA, conditional access and offboarding, and keep one emergency account: ```json { "Beacon": { "UserManagement": { "LocalLogin": { "Enabled": false, "BreakGlassUserNames": [ "superadmin" ] } } } } ``` With `Enabled: false`: - `POST /beacon/api/auth/login` is **not mapped**, unless a break-glass account is listed. Then every name gets the same password check (so timing reveals nothing), and only a stored, enabled, internal user whose stored user name is on the list signs in, whichever provider authenticated the password. Other accounts get the generic answer and the attempt logs `PasswordSignInRefused` (7120), without the name typed. - The login page shows only the single sign-on button. A break-glass account opens the password form at `/beacon/login?local=1`. - A break-glass sign-in logs `BreakGlassSignIn` (7119) at Warning. Treat every one as an incident. `BreakGlassUserNames` holds up to 3 names, each compared exactly (case included) with the stored user name; the holder may type that name or the account's e-mail address. A blank entry or a longer list fails startup, and so does turning local login off with no OIDC, no `BeaconApiOptions.ExternalLogin` (a host-owned login page) and no break-glass account. The JWT login API (`ExternalLoginEndpoint`) does not count for this check. The first-run super admin is a password account: list it (or another internal Admin) before turning local login off. Password sign-in, and the first-run setup, are throttled per client address: 10 login attempts per 60 seconds, then `429` with `Retry-After`. Behind a reverse proxy configure forwarded headers, or the limit applies to the proxy's address. After 5 wrong passwords an internal user is locked out for 15 minutes: every password, the right one included, gets the generic answer, and an administrator's password reset ends the lock early. A wrong current password on a password change counts too, and a locked user cannot change the password. The count starts again after 15 minutes without a wrong password. Break-glass accounts are never locked, only limited per address. Anyone who knows a user name can lock that user out this way; `Beacon:RateLimits:Lockout:Enabled` `false` turns the lockout off. Both are set under [`Beacon:RateLimits`](https://moberghr.github.io/beacon/getting-started/configuration/#request-limits). ## Passwords set by an administrator Whoever sets a password can sign in with it, so a password an Admin sets (a new internal user, or a reset through `IUserManagementService.ResetPasswordAsync` in host code) must be replaced at the user's first sign-in. There is no REST route or UI action that sets another user's password. - The right user name and password answer `401` with `passwordChangeRequired: true` (no session); a wrong password answers as usual. - The login page asks for a new password and sends the same sign-in again with `newPassword`. When the rules accept it (and it differs from the one set), it becomes the user's password, the requirement clears, the security stamp changes and the user is signed in. A refused password answers `400` with the reason. - Changing your own password (`POST /beacon/api/user-settings/change-password`) also clears the requirement. ## Password hashes Passwords are hashed with PBKDF2-HMAC-SHA256, a 32-byte random salt per password and 600,000 iterations. The iteration count is stored in the hash itself (`v2$600000$` in `PasswordHash`; the salt stays in `PasswordSalt`), so no schema change is needed to raise it. - A hash written by an earlier version (bare Base64 at 100,000 iterations) or with fewer iterations still verifies. The user's next successful password sign-in replaces it with a hash at the current count. That is the same password, so the user's sessions stay. The replacement is written only while the stored hash is still the one verified, so it never undoes a password change made at the same moment. - A malformed stored hash, or one claiming fewer than 100,000 or more than 10,000,000 iterations, verifies as a wrong password and logs a warning. - Every verification costs the current 600,000 iterations: an older hash is topped up with discarded work, and an unknown user, a missing or a malformed hash costs a full dummy derivation, so the time a failed sign-in takes does not tell these accounts apart. - **Rollback:** a version before this one cannot read a `v2$` hash, so users whose hash was upgraded cannot sign in with a password after a rollback until an Admin resets their password. - A host's own `IPasswordHasher` keeps working: `NeedsRehash` defaults to `false`, so Beacon never replaces its hashes. ## Sign-in sessions A `Beacon.Auth` session is checked on the server, not only by its cookie: - **Revalidation.** With user management on, a session is checked against the user store at most every `Beacon:Authentication:Session:RevalidateMinutes` (default `5`; `0` checks every request). The user must exist, be enabled, not archived, and carry the security stamp the session signed in with; otherwise the session is signed out (`401` on the API). The session's roles are replaced by the stored roles (plus any identity-provider mapped permissions within their lifetime). A host without user management keeps its own principals: only the absolute lifetime applies. - **Security stamp.** It changes when a password is changed or reset, when the user is disabled, re-enabled or archived, when their roles, user name or e-mail address change, and on **Sign out everywhere**. Changing your own password keeps the current session and ends the others. - **Sign out everywhere.** An Admin, or a UserAdmin for a user they may change, ends every session of a user from the user dialog (`POST /beacon/api/users/{id}/sign-out-everywhere`). A user ends all their own sessions from the account menu (`POST /beacon/api/auth/logout?everywhere=true`), offered when `/auth/me` reports `signOutEverywhereEnabled`. When the sessions cannot be signed out (user management off, or an API key or MCP caller) the answer is `409`. API keys are not affected. - **Absolute lifetime.** A session ends `Beacon:Authentication:Session:AbsoluteLifetimeHours` (default **12**) after sign-in, however often sliding expiration or "remember me" renews its cookie. The requested cookie lifetimes (`options.Authentication.CookieExpirationHours`, default 24, and `RememberMeExpirationDays`, default 30, for "remember me") are capped by it. - **Single sign-on cookie.** A single sign-on session gets a browser-session cookie: closing the browser ends it, and the next visit signs in through the identity provider again. `Beacon:Authentication:Session:PersistSsoCookie: true` keeps the cookie until it expires instead, within the absolute lifetime. - **Realtime.** A SignalR connection is checked when opened and closed when the session's next check is due (at least one minute); the shell reconnects through a new check. ```json { "Beacon": { "Authentication": { "Session": { "RevalidateMinutes": 5, "AbsoluteLifetimeHours": 12 } } } } ``` Both are validated at startup (`RevalidateMinutes` 0–1440, `AbsoluteLifetimeHours` 1–8760). A session a check ends logs `SessionEnded` (7118); a sign-out everywhere logs `UserSessionsRevoked` (7117). The cookie is `HttpOnly`, `SameSite=Lax` and `Secure` when the request is HTTPS. It is protected by ASP.NET Core Data Protection, which belongs to the host: see [Configuration → Data Protection](https://moberghr.github.io/beacon/getting-started/configuration/#data-protection-the-keys-behind-the-session-cookie), including what to do after restoring a Beacon database into another environment. ## Who assigned a role Every `user_roles` row records when the role was assigned (`AssignedAt`) and who assigned it (`AssignedByUserId`, the assigner's external id): when an Admin or UserAdmin creates a user, changes a user's roles, or calls `IUserManagementService.AssignRoleAsync`. It is empty where Beacon grants the role itself: the default role at a first SSO or MCP sign-in, and the super admin's Admin role at setup. Identity-provider mapped permissions are never stored. ## Owned resources Before a user leaves, find what they hold: open the user on the **User Management** page and choose **Show owned resources** (an Auditor or a UserAdmin; nothing is read until you do). It lists, one kind per tab and ten to a page, the live queries, subscriptions, data contracts, recipients and migration jobs the user **owns** (the user whose access scheduled runs act with; recipients and migration jobs have no owner), **created**, or **changed last**, with the dates. Reassign what they own (see [Subscriptions: Change the owner](https://moberghr.github.io/beacon/features/subscriptions/#change-the-owner)) before you disable them, since scheduled runs of a disabled owner are skipped. Archived resources are not listed; an archived user is found too. The same list is `GET /beacon/api/users/{id}/owned-resources?kind={kind}` (paged and sortable like other lists, oldest first; `404` for an unknown user). `kind` is a number: `0` queries, `1` subscriptions, `2` data contracts, `3` recipients, `4` migration jobs. Each item carries `id`, `kind`, `name`, `isOwner`, `isCreator`, `isLastModifier`, `nameWithheld`, `createdTime` and `modifiedTime`. Who created or last changed a resource is recorded from the version that added the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/#who-created-and-changed-what) on; resources from before show neither. The list crosses projects, so names follow what the caller may read: an Auditor sees every name; a UserAdmin sees the names of the resources its projects reach (as the resource's own list shows them; recipients belong to no project and are always named), and every other resource by id and kind only: `name` empty and `nameWithheld: true`, shown as *Name withheld*. Every read is recorded first in the security audit trail (`identity.owned_resources_read`, the kind and the page) and fails when that record cannot be written. ## Recorded in the security audit Sign-ins (password, single sign-on and break-glass, successful, failed and refused), sign-outs, sessions Beacon ended, first-run setup (the super admin, and setup tokens refused), users created, provisioned, updated, enabled, disabled and archived, passwords changed or set by an administrator, and role changes are recorded in the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/#event-catalogue), by user id, never by name, e-mail address, the sign-in name typed or an external id. A failed password sign-in names the user the sign-in name resolved to, when it did, and the attempt by a keyed hash of the name typed, so many attempts on one name can be told from attempts spread over many names without the name being stored. A right password refused because an administrator set it and the user must choose a new one is recorded as a refused sign-in (`password_change_required`). An Auditor reads them on the **Security Audit** page. ## User entity | Field | Description | |---|---| | `ExternalId` | a GUID for internal users; the token subject for external users | | `UserName` | unique user name | | `Email`, `DisplayName` | optional | | `IsInternalUser` | the password is stored in Beacon | | `PasswordHash` | PBKDF2-SHA256 hash, written as `v2$$`; null for external users | | `IsSuperAdmin` | passes every check | | `IsEnabled` | enabled or disabled | | `LastLoginAt` | last sign-in with the user's own identity (password, SSO, an identity-provider token; never an API key) | | `SecurityStamp` | changes whenever every session of the user must end | | `MustChangePassword` | the user must choose a new password before signing in | | `UserRoles` | assigned roles, with `AssignedAt` and `AssignedByUserId` | The `users`, `roles` and `user_roles` tables live in the `beacon` schema and are created by Beacon's migrations. ## Routes All routes are under `/beacon/api`. The OpenAPI document is at `/openapi/v1.json`; see also [Routes & endpoints](https://moberghr.github.io/beacon/reference/routes/). ### Sign-in and the current user | Route | Description | |---|---| | `POST /auth/login` | Sign in with user name and password. Needs the antiforgery token (`X-XSRF-TOKEN`, from `GET /csrf`), `400` without it. `401` with `passwordChangeRequired: true` when an Admin set the password (send `newPassword`). Not mapped while local login is off and no break-glass account is listed. Throttled per address | | `POST /auth/logout` | End the current session; needs the antiforgery token (`X-XSRF-TOKEN`), `400` without it. `?everywhere=true` ends every session of the user | | `GET /auth/me` | The current user, and flags such as `realtimeEnabled` and `signOutEverywhereEnabled`. Anonymous | | `GET /auth/permissions` | `canRead`, `canWrite`, `isAdmin` and the functional `permissions` held now | | `GET /auth/sso` | Whether SSO is offered, its button label and `localLoginEnabled`. Anonymous | | `GET /auth/sso/challenge` | Starts the SSO sign-in | | `GET /csrf` | Issues the antiforgery token for a browser session (`400` for callers authenticated by an `Authorization` header) | | `GET /user-settings` | The caller's own settings | | `POST /user-settings/change-password` | Change your own password (needs the current one); ends your other sessions | ### Users For an Admin or a UserAdmin (see [Functional roles](#functional-roles) for a UserAdmin's limits). | Route | Description | |---|---| | `GET /users` | the user directory | | `GET /users/roles` | every role, with the permission each functional role grants | | `POST /users/internal` | create an internal user with roles (Admin only) | | `POST /users/external` | create an external user with roles | | `PUT /users/{id}` | change a user's display name, e-mail address, user name or enabled state (user name and e-mail address: Admin only) | | `POST /users/{id}/toggle-enabled` | enable or disable a user | | `PUT /users/{id}/roles` | replace a user's roles (ends their sessions when they change) | | `POST /users/{id}/sign-out-everywhere` | end every session of the user (`204`; `409` when user management is off) | | `GET /users/{id}/owned-resources` | the user's live resources of one `kind`, paged; for an Auditor or a UserAdmin (see [Owned resources](#owned-resources)) | ### First-run setup | Route | Description | |---|---| | `GET /setup/status` | whether first-run setup is needed | | `POST /setup/superadmin` | create the super admin (first run only; needs `setupToken`, `403` when missing or wrong) | | `GET /setup/roles` | the roles, anonymous only while setup is open | ## Troubleshooting | Symptom | Check | |---|---| | No login page | `EnableLoginForm = true`, `AddBeaconCookieAuthentication("/beacon")`, the login endpoints and middleware mapped; the page is `/beacon/login` | | Setup page does not appear | it appears only while no super admin has ever existed (an archived one counts); other users never close it | | Setup token rejected | the configured token, or the one printed to the process console of the replica that served the page | | External user gets `401` | the user exists, is enabled and is external; issuer and audience configured; the token is an access token. The server logs a reason code (`NoBeaconUser`, `DisabledUser`, `IdToken`, `NotAccessToken`, `NotAdmitted`) with tenant, issuer and a subject hash | | `NotAccessToken` from a non-Entra provider | set `AccessTokenClaim` to a claim the provider puts in access tokens only | | SSO user sees "no access yet" | assign a role, or set `DefaultRoleName`; the user then selects **Check again** | | Login answers `429` | 10 attempts per 60 seconds per address; behind a proxy, configure forwarded headers | | The right password is refused for a while | The user is locked out after 5 wrong passwords (15 minutes, `auth.user_locked_out` in the security audit); wait, or reset the password | ## Upgrading from 4.5 - **First-run setup needs a token** (`Beacon:UserManagement:SetupToken`, at least 32 characters, or the one printed to the process console). Automation calling `POST /beacon/api/setup/superadmin` must send `setupToken`. Setup is "no super admin has ever existed", and until then SSO and MCP provisioning create no user. - **SSO requires an admission decision.** With OIDC enabled set `AllowedTenants` or `AllowAnyTenant: true`, or the host does not start. Guests are blocked by default. New SSO users get no role unless `DefaultRoleName` is set. Review the users earlier versions provisioned with `Viewer`. - **JWT validation is mandatory and strict**, tokens must prove they are access tokens (set `AccessTokenClaim` if your provider's access tokens carry neither `typ: at+jwt` nor an Entra client claim), and REST bearer tokens must name an existing, enabled, external Beacon user. - **Sign-out** is `POST /beacon/api/auth/logout` with the antiforgery token; `GET /beacon/api/auth/signout` was removed, and the OIDC front-channel sign-out path is served only with `EnableFrontChannelLogout: true`. - **Disabling, re-enabling and archiving revoke API keys.** - **Sessions are checked on the server** and end 12 hours after sign-in by default; "remember me" no longer outlasts the absolute lifetime. The `AddUserSecurityStamp` migration adds `security_stamp` and `must_change_password`. - **New projects are restricted**, and a stored user is an Admin by current roles only. See [Authorization → Upgrading](https://moberghr.github.io/beacon/security/authorization/#upgrading-from-45-project-access). - **The encryption key is checked in every environment**, and **Data Protection is the host's** (call `AddBeaconDataProtection` or configure your own; everyone signs in again once). - **Local login stays on** by default; with SSO, turn it off and list a break-glass account. - **Passwords an Admin sets must be replaced at first sign-in**; a custom login client must handle `passwordChangeRequired` and send `newPassword`. - **Functional roles** are created when roles are next listed; a role of yours with one of their names grants nothing until renamed. - **Public API.** `IUserManagementService.GetOrCreateExternalUserAsync` takes a nullable `defaultRoleName`; `CompleteRequiredPasswordChangeAsync` is a new default member; `LoginRequest` gains `NewPassword` and `LoginResponse` `PasswordChangeRequired` (positional records: recompile); `CreateInternalUserRequest`/`CreateExternalUserRequest` gain `AssignedByUserId`. ## See also - [Authorization](https://moberghr.github.io/beacon/security/authorization/): project access and functional permissions - [API keys](https://moberghr.github.io/beacon/security/api-keys/) - [Admin settings](https://moberghr.github.io/beacon/security/admin-settings/) - [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/): every sign-in key --- # API Keys > Scoped, SHA256-hashed API keys for programmatic access, with Read/Execute scopes, a project list, a mandatory expiry, and revocation tied to the owner's account. Source: https://moberghr.github.io/beacon/security/api-keys/ API keys give programmatic clients (CI pipelines, monitoring bots, and MCP clients) scoped access to Beacon without an interactive login. Keys are SHA256-hashed at rest, carry explicit scopes, are confined to the projects they name, always expire, and stop working when their owner's account is disabled or removed. ## Overview Every API key is: - **Scoped**: a key carries one or both scopes, `Read` and `Execute`, that limit what it can do - **Hashed at rest**: only a SHA256 hash of the key is stored; the raw key is shown exactly once at creation - **Bound to a user**: keys are created by a signed-in user and act for that user, never with more than that user may do; when the user is disabled or archived, the key stops working and is revoked - **Revocable**: by its owner or by an administrator; revoking takes effect immediately on the next request - **Expiring**: every new key has an expiration date: 90 days by default (or `Beacon:ApiKeys:MaxLifetimeDays`, if lower), at most `MaxLifetimeDays` (365 by default) - **Project-restricted**: a new key names at least one project and is confined to those projects (keys created before lists were required reach their owner's projects) A key looks like this: ``` sk-sem_ ``` The random part is 32 bytes from a cryptographically secure generator, base64-encoded with `+`, `/` and `=` removed, so its length varies slightly (at most 43 characters). Beacon stores its SHA-256 hash and a 16-character display prefix. ## Creating a Key 1. Go to **API Keys** in the left navigation (`/beacon/api-keys`) 2. Click **Generate key** 3. Fill in the dialog: | Field | Required | Description | |-------|----------|-------------| | **Key name** | Yes | A descriptive label, e.g. `CI Pipeline` | | **Scopes** | Yes (at least one) | `Read` and/or `Execute`. See [Scopes](#scopes). `Execute` needs the Editor role or above | | **Projects** | Yes (at least one) | The projects the key may use. See [Project restrictions](#project-restrictions). The picker lists only the projects you can read without the Admin bypass | | **Expiration date** | No | Leave empty for 90 days (or `MaxLifetimeDays`, if lower). A date further away than the configured maximum (365 days by default) is refused | 4. Click **Generate key**: the raw key is displayed once :::caution **The raw key is shown exactly once.** Beacon stores only a SHA256 hash. The plaintext key cannot be recovered later. Copy it immediately and store it in a secrets manager. If you lose it, revoke the key and generate a new one. ::: The dialog blocks accidental dismissal (backdrop click and Escape are disabled) until you confirm you have copied the key. **Who may create which key:** - Any signed-in user with read access (Viewer and above) can create `Read` keys and revoke their own keys. - `Execute` keys need write permission: the Editor role or above (or a super admin). A key never does more than its owner may: the `Execute` scope is checked against the owner's roles again on every request (see [Scopes](#scopes)). - A disabled user cannot create keys. - Keys are managed from a signed-in browser session only. A request authenticated with an API key, as an MCP caller or with a REST bearer token gets `403` on every `/beacon/api/api-keys` route (listing, creating and revoking keys, and the administrator routes). ### Project restrictions Every new key names the projects it may use: `allowedProjectIds` on the create request (`POST /beacon/api/api-keys`) must list at least one project (`400` otherwise), and each must be a project the creator can read **without the Admin bypass**. An open project, or a restricted one they are a member of (`404` otherwise). An Admin's key cannot name a restricted project the Admin is not a member of. The dialog's project picker lists only these projects (`GET /beacon/api/projects?forApiKey=true`). A key never exceeds its owner: on every request it reaches the projects in its list **where its owner still has access**, at most at the owner's role there, capped by the lower of the key's scope and the owner's global role (a `Read` key reads and runs saved SQL only; a key whose owner may neither read nor write reaches nothing). A key never gets the Admin bypass, even when its owner is an Admin, and never manages project members. See [Project access](https://moberghr.github.io/beacon/security/authorization/#project-access). The list is enforced on every REST route and on the [MCP Server](https://moberghr.github.io/beacon/mcp/server/), by the same project access rules on both, fail-closed: a malformed restriction claim denies every project rather than falling open. **Keys created without a list** (before lists were required) act with their owner's project access, over REST and on MCP: they reach every project their owner may use. Re-issue them with a project list to confine them. ## Scopes | Scope | Grants | |-------|--------| | `Read` | Read-only requests: `GET` routes under `/beacon/api` that list or read stored data (configs, reports, stored results) | | `Execute` | Everything else: every request that changes state, runs SQL, dials a data source, makes an outbound call or calls the LLM, and the MCP endpoint | **How enforcement works:** - For a key, every `/beacon/api` request that is not `GET`, `HEAD` or `OPTIONS` needs the `Execute` scope: creating or editing queries, subscriptions and recipients, running previews, evaluating data contracts, triggering scans, documentation generation, AI actors and so on. The few `GET` routes that run SQL or dial a data source (for example `GET /beacon/api/data-sources/{id}/metadata`, which scans the data source when no metadata is stored yet) need `Execute` too. Everything else a `Read` key may call is a read-only `GET`. - This is enforced on every request, whether or not `Authorization.Enabled` or `UserManagement.Enabled` is set. When either is set, the owner's role is checked as well: the scope is a ceiling, not a grant. - **`Execute` follows the owner.** A key keeps the `Execute` scope only while its owner has write permission (the Editor role or above, or super admin, and the authorization provider's consent), checked on every request from the owner's current roles and the provider's current answer. When the owner loses it, the key acts as a `Read` key (and a warning naming the key id is logged once per process); when the owner regains it, so does the key. This holds whatever `Authorization.Enabled` says. - The **MCP endpoint** `/beacon/mcp` requires the `Execute` scope as a whole, because its tools can run SQL. A `Read` key cannot open an MCP session when the host maps the endpoint with `app.MapBeaconMcp()`, and inside the MCP layer every request from it is refused either way. See the [MCP Server guide](https://moberghr.github.io/beacon/mcp/server/). - The **real-time hub** (`/beacon/api/hub`) accepts browser sessions only; no key reaches it. - Routes that require the Admin role or a [functional permission](https://moberghr.github.io/beacon/security/authorization/#functional-permissions) (user management, settings, data-source administration, the key administration below) are never reachable with a key: keys carry no role and no functional permission, whatever their owner holds. A key refused on a route that requires a permission gets `403` with the detail *"API keys cannot use administrative routes; sign in as a user with the X permission."* Automate administrative work as a signed-in user. - Scopes are attached to the request as claims by the API-key authentication middleware. Interactive browser sessions (cookie/OIDC) carry no scope claims and are not scope-gated. They are governed by user roles instead. Scopes constrain **API keys and Entra callers on MCP only**. - Requests authenticated with an API key or a bearer token in the `Authorization` header need no antiforgery token, and `GET /beacon/api/csrf` does not issue one to them (it answers `400`). Browser sessions keep sending the token as before. :::note Grant the narrowest scope that works. A monitoring integration that only reads configuration and reports needs `Read`; reserve `Execute` for clients that actually run queries or change things. ::: ## Managing Keys The **API Keys** page (`/beacon/api-keys`) lists your keys with: | Column | Description | |--------|-------------| | **Name** | The label you gave the key | | **Prefix** | The first 16 characters (e.g. `sk-sem_AbCd12345…`), enough to identify a key without exposing it | | **Scopes** | The scopes the key grants (`Read`, `Execute`) | | **Projects** | The projects the key is restricted to, or an empty value for a key created before project lists were required | | **Created** | Creation timestamp | | **Last used** | Timestamp of the most recent authenticated request, or `Never` | | **Expires** | The expiration date, `Never` (keys created before expiry became mandatory), or an `Expired` badge | | **Status** | `Active` while the key works; otherwise `Inactive`: revoked, expired, or its owner disabled or archived (the administrators' table says `Revoked` for a revoked key) | ### Revoking a key Click **Revoke** next to an active key and confirm. Revocation is immediate. Any integration using that key stops working on its next request. Revoked keys remain in the list (marked `Revoked`) for auditability; they are never silently deleted. ### Administering every user's keys Admins, Auditors and UserAdmins see a second table on the same page, **All API keys**: every user's keys with their owner. Click an owner to show only that user's keys; Admins and UserAdmins **Revoke** an active key (a UserAdmin who is not an Admin is not offered the keys of an Admin, a super admin or a functional role holder, other than their own). The same is available over the API; each listed key's `canRevoke` says whether the caller may revoke it: | Method | Route | What it does | Who | |--------|-------|--------------|-----| | `GET` | `/beacon/api/api-keys/admin?userId={id}` | Every key, newest first, or only the keys of user `id`; paged and sortable like other lists | Admin, Auditor, UserAdmin | | `DELETE` | `/beacon/api/api-keys/admin/{id}` | Revokes key `id`, whoever owns it (a UserAdmin who is not an Admin: not the key of an Admin, a super admin or a functional role holder, `403`) | Admin, UserAdmin | Besides the route's policy, the caller must hold the role in the user store at the time of the request (enabled; super admin, the Admin role, or the [functional permission](https://moberghr.github.io/beacon/security/authorization/#functional-permissions)). A session whose user has since been demoted or disabled is refused with `403`. API keys never reach them. ### When the owner leaves A key works only while its owner can sign in: - **Disabling a user** (the toggle on the Users page, or saving the user as disabled) revokes all of that user's keys in the same save, and again on every later save of the user as disabled. - **Enabling the user again does not bring any key back:** the re-enable itself revokes every key of theirs that is still active (for example a key created while they were being disabled, or a key of a user disabled before keys followed their owner). Issue new keys. - **Archiving (deleting) a user** revokes their keys the same way. - A key whose owner is disabled, archived or missing is rejected on every request, as is a key with no owner. Disabling, archiving and re-enabling also advance the owner's API-key generation, so a key created while one of those changes was being saved is rejected as well. - **A role removed from the owner** takes effect on their keys at the next request: a key is capped by its owner's current global role, and keeps `Execute` only while the owner may still write (see [Scopes](#scopes)). - **An owner who no longer signs in** (opt-in): with `Beacon:ApiKeys:MaxOwnerInactivityDays` set, a key stops working once its owner has not signed in to Beacon with their own identity (a password, a single sign-on, or an identity-provider token on MCP or the REST API, never an API key) for that many days (counted from the account's creation for an owner who never signed in). This covers an owner removed at the identity provider but never disabled in Beacon. The caller gets `401` with *"This API key is inactive: its owner has not signed in to Beacon for N days. The key works again after the owner signs in."*, the refusal is logged with the key id (`owner_inactive`), and the key is listed as inactive. Nothing is revoked: the key works again after the owner signs in. Off by default (`0`). Every sign-in with the owner's own identity counts, including an MCP client or agent that calls `/beacon/mcp` or the REST API with the owner's delegated Entra token, so an agent acting for the owner keeps their keys active as well. Offboard at the identity provider (no new tokens) and disable the Beacon user to end the keys, sessions and delegated access together. An account used only through its API keys (a service account) never signs in: with the rule on, its keys stop working that many days after its last sign-in, or its creation, unless it signs in now and then. ### Recorded in the security audit Creating a key (`keys.api_key_created`: its owner, scopes, projects and expiry, never the key or its hash), revoking your own (`keys.api_key_revoked`) and revoking someone else's (`keys.api_key_revoked_by_administrator`) are recorded in the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/), as are the keys revoked when their owner is disabled, re-enabled or archived (counted on the user's event). Every other event recorded while a request used a key names the key (`ApiKeyId`) and its owner (`ActorUserId`), with the actor kind `api_key`. An Auditor reads them on the **Security Audit** page. ### Expiration Every new key expires: after 90 days when no date is given (or after `MaxLifetimeDays`, if that is lower), and never later than `Beacon:ApiKeys:MaxLifetimeDays` days after creation (365 by default; see [Configuration → API Keys](https://moberghr.github.io/beacon/getting-started/configuration/#api-keys)). A date that is not in the future, or beyond the maximum, is refused with `400`. A key stops working at its expiry instant. An expired key fails validation exactly like a revoked one: the caller receives `401 Unauthorized` with an "Invalid or expired API key" problem response. Expired keys stay visible in the list with an `Expired` badge. A key created before expiry became mandatory may have no expiration date. Such a key works until it is revoked, and logs a warning with its key id the first time it is used after a restart. Rotate these keys. To retire them all at once, set `Beacon:ApiKeys:EnforceMaxLifetimeOnExistingKeys` to `true`: such a key then expires `MaxLifetimeDays` after it was created (no stored data changes; unset the option and they work again). ## Using a Key API keys are passed in the standard `Authorization` header as a Bearer token: ``` Authorization: Bearer sk-sem_YOUR_API_KEY ``` This is the **only** accepted format. There is no `X-Api-Key` header. The authentication middleware only engages when the header starts with `Bearer sk-sem_` (compared case-insensitively); anything else falls through to the other authentication schemes (cookies, JWT). ### With the REST API Read endpoints under `/beacon/api/*` accept a `Read` key directly; anything that changes state or runs SQL needs an `Execute` key. No antiforgery token is needed: ```bash # List projects (Read) curl "https://your-beacon-host/beacon/api/projects" \ -H "Authorization: Bearer sk-sem_YOUR_API_KEY" # List saved queries (Read) curl "https://your-beacon-host/beacon/api/queries" \ -H "Authorization: Bearer sk-sem_YOUR_API_KEY" ``` The full endpoint surface is described by the OpenAPI document at `/openapi/v1.json`. The `/beacon/api/api-keys` routes are the exception: keys are managed from a signed-in browser session only. ### With the MCP Server API keys are the authentication mechanism for the [MCP Server](https://moberghr.github.io/beacon/mcp/server/) at `/beacon/mcp`. Use an `Execute` key and add it to your MCP client configuration: ```json { "mcpServers": { "beacon": { "url": "https://your-beacon-host/beacon/mcp", "headers": { "Authorization": "Bearer sk-sem_YOUR_API_KEY" } } } } ``` If the key is restricted to a single project, all MCP tools resolve that project automatically. If it has access to multiple projects, pass `project_id` in tool calls. See [MCP Server](https://moberghr.github.io/beacon/mcp/server/) for the full tool reference. ## How Validation Works Understanding the mechanics helps when debugging authentication failures: 1. **Header check**: the middleware looks for `Authorization: Bearer sk-sem_...`. Requests already authenticated by another scheme (e.g. a browser cookie session) skip API-key processing entirely. 2. **Hash lookup**: the presented key is SHA256-hashed and matched against the stored hash. The plaintext key never touches the database; the stored 16-character prefix is for display in the UI only and plays no part in validation. 3. **Status checks**: a matching key is rejected if it has been revoked, if its expiry instant has been reached, or if its owner is missing, archived or disabled, or has moved to a later API-key generation than the key's; so is a key whose stored scopes leave no valid scope (none, or only unknown values). All of these return the same `401` problem response: "Invalid or expired API key." The reason is logged at Warning with the key id (`revoked`, `expired`, `owner_disabled`, `owner_archived`, `owner_missing`, `owner_generation_changed`, `no valid scopes`). Never the key or its hash. With `MaxOwnerInactivityDays` set, a key whose owner has not signed in for that long is refused too (`owner_inactive`), with its own `401` message saying so. 4. **Bookkeeping**: the key's *Last used* timestamp is updated. 5. **Identity**: a claims identity is built from the key: its scopes (only `Read` and `Execute`; a key stored with the retired `Admin` scope gets `Execute`, and `Execute` reads as `Read` while the owner has no write permission by their roles or the authorization provider), its project restrictions, and the owner. The scope filter, authorization policies and, when enabled, the owner's role then decide per request. ## Security Best Practices - **Use least scope.** Create separate keys per integration, each with only the scopes it needs. A read-only reporting job should never hold an `Execute` key. - **Name only the projects you need.** Every key names its projects; if an integration only works with one project, list only that one. A leaked key exposes the projects it names, not all of them. - **Keep lifetimes short.** Prefer the shortest expiry that works; lower `Beacon:ApiKeys:MaxLifetimeDays` if your policy demands it. A forgotten key that expires is harmless. - **Rotate regularly.** There is no in-place rotation: generate a new key, switch the integration over, then revoke the old one. The *Last used* column tells you when the old key has actually gone quiet. - **Never commit keys.** Keys must not appear in source code, config files under version control, CI logs, or chat. Inject them via environment variables or a secrets manager. If a key is ever committed, even to a private repository, revoke it immediately. - **Revoke on suspicion.** Revocation is instant and free. If there is any doubt a key has been exposed, revoke first and re-issue after. - **Audit periodically.** Administrators: review **All API keys** for keys with `Never` or stale *Last used* values, keys that never expire (created before expiry became mandatory), and keys of people who left, and revoke what is no longer needed. :::note Beacon never stores or logs the plaintext key. If you find yourself needing to "look up" an existing key's value, that is by design impossible. Generate a new key instead. ::: ## Troubleshooting **`401 Unauthorized`: "Invalid or expired API key"** The key was recognized as an API key but failed validation. Check that it has not been revoked, has not passed its expiration date, that its owner's account is still enabled, and that it was copied in full (keys are long, and a truncated paste is the most common cause). The server log names the key id and the reason. **Request behaves as unauthenticated** The middleware only engages on headers that begin with `Bearer sk-sem_`. Verify the header is exactly `Authorization: Bearer sk-sem_...`. A missing `Bearer ` prefix or a custom header name (e.g. `X-Api-Key`) is silently ignored. **`403`: "This operation requires the Execute scope."** The request changes state, runs SQL or calls out, and the key only has `Read`, or it has `Execute` but its owner no longer has the Editor role (or above). Generate a key with the `Execute` scope, or ask an administrator to restore the owner's role. **`403` on `/beacon/api/api-keys`** Keys cannot list, create or revoke keys, and neither can MCP callers or REST bearer tokens. Sign in to the UI. **`403` when creating an `Execute` key** Creating an `Execute` key needs the Editor role or above. Ask an administrator for the role, or create a `Read` key. **`400` when creating a key** The scope is not `Read` or `Execute` (the `Admin` scope can no longer be issued), no scope was given, or the expiration date is in the past or beyond the configured maximum. **"No projects are available to you" via MCP** The key reaches no project: none of the projects in its list is one its owner may still use (for example, a project was restricted and the owner is not a member), or its owner may no longer use any project. Check the key's allowed projects and the owner's access. **"Access denied: you do not have access to project N"** means the same for the one project the call named. **`404` on a project's resources with a key** The project is not in the key's list, or the key's owner may no longer use it (for example, it was restricted and the owner is not a member). ## Upgrading from 4.5 API keys are now scoped on every route, expire, follow their owner's account and are confined to their projects. - **`Read` keys are read-only everywhere.** Every `/beacon/api` request other than `GET`/`HEAD`/`OPTIONS` (and `GET /beacon/api/data-sources/{id}/metadata`) answers `403` to a key without `Execute`, whatever `Authorization.Enabled` and `UserManagement.Enabled` say. Integrations that wrote with a `Read` key need an `Execute` key. - **The `Admin` scope is retired.** New keys carry only `Read` and `Execute`; existing `Admin` keys act as, and are listed as, `Execute`. The MCP discovery documents advertise `scopes_supported: ["Execute"]`. - **`Execute` follows the owner's current write permission**, checked on every request (the owner's roles and the authorization provider's answer). Existing `Execute` keys of Viewers act as `Read` keys until the owner regains write permission. - **Who may create keys.** `Execute` keys need the Editor role or above; Viewers create `Read` keys and revoke their own; disabled users create none. Keys are managed from a signed-in browser session only: every `/beacon/api/api-keys` route answers `403` to a key, an MCP caller or a REST bearer token. - **Every new key expires** (90 days by default, at most `Beacon:ApiKeys:MaxLifetimeDays`). A key without an expiration date from an earlier version warns once per process; rotate it, or set `EnforceMaxLifetimeOnExistingKeys` to expire such keys. - **Keys follow their owner.** Disabling, archiving or re-enabling a user revokes their active keys in the same save and advances their API-key generation, so a key created during the change is refused too. The `AddApiKeyOwnerGeneration` migration (both providers) adds the generation columns; existing keys start at generation 0 and keep working. - **Keys are confined to their projects on REST**, not only on MCP, and new keys must name at least one project (`400`; `404` for a project the creator cannot read without the Admin bypass). Keys without a list act with their owner's access, and now reach their owner's projects on MCP too, where before they reached none. Re-issue them with a project list. - **No antiforgery token for header-authenticated callers**: requests with an API key or bearer token skip it, and `GET /beacon/api/csrf` answers them `400`. - **Map MCP with `app.MapBeaconMcp()`**, which applies the Execute-scope policy (it needs `builder.Services.AddBeaconApiAuthorization()`). The MCP layer also refuses callers without `Execute` on a weaker mapping. - **Administrator routes.** `GET /beacon/api/api-keys/admin` and `DELETE /beacon/api/api-keys/admin/{id}` are new, checked against the caller's stored roles on every request. - **Owner inactivity** (opt-in): `Beacon:ApiKeys:MaxOwnerInactivityDays`. - **Public API.** `IApiKeyService.GenerateApiKeyAsync` requires the user id and scopes and stores the owner generation; `ValidateApiKeyAsync` must refuse a key of another generation and return the credential with its user and roles loaded, and may throw `ApiKeyOwnerInactiveException`. `ApiKeyEntry` gains `AllowedProjectIds` (positional: recompile). The Execute-scope policy name is `BeaconScopes.ExecuteScopePolicyName`. --- # Security audit trail > An append-only, signed record of who signed in, changed identities, keys, access, data sources, delivery, settings, saved queries, approvals, ownership, tasks, curation, data quality and migrations, for an Auditor, verified on a schedule, with a copy on the Beacon.Audit log stream for your SIEM. Source: https://moberghr.github.io/beacon/security/audit-trail/ Beacon records every security-relevant action in one append-only trail: sign-ins and sign-outs, changes to users, roles, API keys and project access, data sources, recipients and subscriptions, settings, saved queries, approvals, ownership, alert tasks, grounding curation, data contracts and migration jobs, and every read of audit data. Each event says who acted, what they did, how it ended and what it concerned, carries a keyed signature over its fields, and is copied to the `Beacon.Audit` log stream once it is committed, so a SIEM holds a copy outside Beacon's database. The trail is read by an **Auditor** (an Admin holds the permission too, see [Functional permissions](https://moberghr.github.io/beacon/security/authorization/#functional-permissions)) on the **Security Audit** page or through the API. It is separate from the [MCP audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/), which records every MCP tool call and the statements it ran. **The database is the source of truth.** The `SecurityAuditEvents` table is the trail; the log-stream copy is a best-effort second copy (see [What the trail does and does not prove](#what-the-trail-does-and-does-not-prove)). ## What an event records Events are stored in the `SecurityAuditEvents` table (`security_audit_events` on PostgreSQL), one row per event. No column has a foreign key: an event outlives the user, key or project it names. | Field | Meaning | |---|---| | `Id`, `EventGuid` | The row id, and a unique id the log-stream copy carries too. | | `CreatedTime` | When the event was recorded (UTC, kept to the microsecond). | | `Category`, `Action` | The event family and what happened, named `{category}.{what happened}` in the past tense; see the [catalogue](#event-catalogue). | | `Outcome` | `success`, `failure` (the attempt did not succeed, such as a wrong password), `refused` (Beacon refused it by rule: local login off, a setup token that does not match, a purge of events that fail verification) or `empty` (there was nothing to act on: a verification that found no event to check, which is not a success). | | `ActorKind` | How the actor was authenticated: `user`, `session` (a signed-in session without a stored Beacon user), `api_key`, `mcp_caller`, `system` (Beacon's own background work), `embed` (an embed grant or token, recorded as the grant's creator) or `anonymous`. | | `ActorUserId` | The acting user's `Users.Id` (an API key's owner); `null` for a caller without a stored Beacon user. | | `ApiKeyId` | The API key the actor used. | | `CallerHash` | For a caller without a Beacon user (a single sign-on subject, a host session, an MCP caller, the name typed at a failed password sign-in): a keyed hash, never the subject or the name itself. For an MCP caller it is the same hash the MCP audit records. | | `ProjectId` | The project the action concerns, when it concerns one. | | `ResourceType`, `ResourceId` | What was acted on: `user`, `api_key`, `project`, `data_source`, `repository`, `query`, `subscription`, `recipient`, `data_contract`, `migration_job`, `ai_actor`, `ai_actor_plan`, `approval_request`, `change_approval_request`, `task`, `settings`, `learned_pattern`, `documentation_patch`, `eval_case`, `schema_relationship`, `glossary_term`, `documentation_section`, `mcp_audit`, `embed_grant` or `security_audit`. | | `DetailsJson` | What else the event needs, as a small JSON object of identifiers, counts, flags, codes and keyed digests (below). | | `TraceId` | The W3C trace id of the request or job that recorded the event. | | `KeyId`, `Hmac` | The integrity key the event was signed with, and the signature; see [Integrity](#integrity-signatures-and-verification). | The details hold identifiers (`addedRoleIds`, `removedRoleIds`, `recipientIds`, `changeApprovalId`, the settings-history ids of a settings change), counts, flags (`secretChanged`, `destinationChanged`, `connectionChanged`, `forced`), times (UTC, to the second) and short codes (a reason, a kind, the keys of the settings that changed, the fields of a data source that changed, a status before and after). People a resource records by their identity-provider id (an owner, a requester, an assignee) are written as their `Users.Id`, or `null` for nobody or an unknown id. A settings change also carries, for each key it changed and in the order of `changedKeys`, a keyed digest of the value before and after (`beforeDigests`, `afterDigests`: 32 hex characters under a key derived from the integrity key; an absent value has a digest of its own). The values never go into the event; the digests let an Auditor tell whether the settings history still holds the values the event was sealed over; a history row edited later no longer matches. A glossary term's and a golden case's events carry their grounding text the same way: `contentFields` names the text fields (a term's `term`, `synonyms`, `definition`, `targetSchema`, `targetTable`, `targetColumn` and `metricExpression`; a case's `question` and `goldSql`), and `beforeDigests` / `afterDigests` hold a keyed digest of each one's stored text in that order: before and after an edit, after a creation or a promotion, before a deletion. The text never goes into the event; the digests show which field an edit changed. A digest is keyed: only a holder of the integrity key can check a disclosed text against it, and digests made under different keys do not compare. ### What is never stored The trail names people by id only and keeps no secret. The details are restricted by construction, not left to care at each event site: - **Never stored:** names, display names and user names, e-mail addresses, the sign-in name someone typed, external (identity-provider) ids, IP addresses, user agents, SQL, questions, comments, notes, descriptions, free-text errors, setup tokens, passwords, API keys, credentials, connection strings, setting values and recipient destinations. A change of a recipient's destination or secret, or of a data source's connection, is recorded as a flag (`destinationChanged`, `secretChanged`, `connectionChanged`), never the value. A single sign-on subject is recorded only as its keyed `CallerHash`; so is the name typed at a failed password sign-in (trimmed and lower-cased, under a key of its own), so attempts on one name can be told from attempts spread over many without the name being stored. - **Codes only.** A text detail is kept only when it is a code (letters, digits and `_ . : -`, at most 64 characters) and its key does not name text (a name, an e-mail address, mail, SQL, a question, a comment, a note, a description, text, a message, a destination, a URL, a token, a secret, a password, a header, a body, an address, an agent, a subject, an external id or a login, in any case and anywhere in the key). Anything else is stored as `[withheld]`, so a mistake at an event site cannot store free text; the Security Audit page shows such a detail as *withheld*. - **Bounded.** A list of ids keeps its first 50 entries and records how many there were (`Count`). The whole details object stays under 2,000 characters: when it would not, every single value is kept and the lists are shortened, the longest first, each one's full length recorded as `Count`, and the details are marked `"truncated": true`. ## Event catalogue Every action belongs to one category, the part of its name before the dot. **Authentication** (`auth`), recorded on their own, since a sign-in has no change to join (see [How events are written](#how-events-are-written)). | Action | When | |---|---| | `auth.password_sign_in_succeeded` | A password sign-in issued a session. | | `auth.password_sign_in_failed` | A password sign-in did not succeed (`failure`); names the user the sign-in name resolved to, when it did, and the attempt by a keyed hash of the name typed. | | `auth.password_sign_in_refused` | A password sign-in refused by rule (`refused`): local login is off and the account is no break-glass account (`local_login_off`), or the password was right but an administrator set it and the user must choose a new one first (`password_change_required`), or the user is locked out after too many wrong passwords, whatever password was sent (`locked_out`). | | `auth.break_glass_signed_in` | A break-glass account signed in with its password while local login is off. | | `auth.signed_out` | A session signed out. | | `auth.signed_out_everywhere` | Every session of a user was ended (**Sign out everywhere**, by the user or an administrator). | | `auth.session_ended` | Beacon ended a session: refused at sign-in or at a check against the user store (the user was disabled, archived or their sessions revoked), with the reason. | | `auth.sso_signed_in` | A single sign-on was admitted. | | `auth.sso_refused` | A single sign-on was refused, with the reason; the subject only as its keyed hash. | | `auth.setup_token_refused` | First-run setup was attempted with a setup token that does not match. | | `auth.user_locked_out` | An internal user's password sign-in was locked after too many wrong passwords in a row (`Beacon:RateLimits:Lockout`); names the user, with the number of attempts, the lockout length in minutes and when it ends. | **Identity** (`identity`) | Action | When | |---|---| | `identity.super_admin_created` | First-run setup created the super admin. | | `identity.user_created` | An administrator created a user. | | `identity.user_provisioned` | Single sign-on created a user at their first sign-in. | | `identity.user_updated` | A user's account changed (which fields changed, as flags, and whether their sessions and API keys ended with it). | | `identity.user_enabled`, `identity.user_disabled`, `identity.user_archived` | A user was enabled, disabled or archived (with how many of their API keys were revoked). | | `identity.last_super_admin_change_refused` | Disabling or archiving the last (enabled) super admin was refused; the event is `refused` and carries the attempt as a code. | | `identity.password_changed` | A user changed their own password. | | `identity.password_reset` | An administrator set a user's password. | | `identity.roles_changed` | A user's roles changed (role ids added and removed). | | `identity.owned_resources_read` | Someone read what a user owns, created or changed last (the kind and the page). | **API keys** (`keys`): `keys.api_key_created`, `keys.api_key_revoked` (by its owner) and `keys.api_key_revoked_by_administrator`. **Project access** (`access`): `access.project_created`, `access.member_added`, `access.member_role_changed`, `access.member_removed`, `access.access_mode_changed` (a project opened or restricted), `access.data_sources_linked`, `access.data_source_unlinked`, `access.repository_token_changed` (whether a token is now set, never the token), `access.repository_scan_requested`, `access.repository_scan_ran` (a background scan ended: its task, the ids of the repositories it scanned and how many failed, were partial or were not scanned; `refused` with the reason when the task was skipped). Events recorded before dashboards were removed keep their `access.dashboard_shared` and `access.dashboard_share_revoked` actions and the `dashboard` resource type, and the page still filters by them. The project-access log events (`ProjectAccessEvents`, 7101 onwards) are still written as before; refused and hidden reads of a project (7108, 7109) and skipped scheduled work (7111) stay on the log only, for their volume. **Data sources** (`data_sources`): `data_sources.data_source_created`, `data_sources.data_source_updated` (which fields changed, by name), `data_sources.credentials_rotated` (the connection changed: its credentials, host or database; never a value), `data_sources.data_source_deleted`, `data_sources.metadata_refreshed` (a forced refresh someone started, and whether they were exempt from its cooldown), `data_sources.migration_writes_changed`, `data_sources.data_source_disabled` and `data_sources.data_source_enabled` (the data source id only). An edit that changes the connection is recorded as `credentials_rotated` only, with the other fields it changed; both edit events carry `targetChanged`, whether the connection now points at another engine, host, port or database (never which). **Delivery** (`delivery`): `delivery.recipient_created`, `delivery.recipient_updated` (which fields changed, whether the destination or the secret changed, and the change request that governs it), `delivery.recipient_deleted`, `delivery.subscription_created`, `delivery.subscription_updated`, `delivery.subscription_archived`, `delivery.subscription_reactivated`, `delivery.subscription_recipients_added`, `delivery.subscription_recipient_removed` and `delivery.subscription_sla_changed` (the SLA hours before and after). **Settings** (`settings`): one event per save that changes something (`settings.mcp_settings_changed`, `settings.mcp_project_settings_changed` and `settings.admin_settings_changed`), naming the keys that changed, the settings-history rows that hold the values before and after, and a keyed digest of each value (above). The values themselves never go into the event; an Auditor reads them in the settings history. **Saved queries** (`queries`): `queries.lock_changed` (a query locked or unlocked) and `queries.mcp_tool_changed` (a query exposed to MCP callers as a tool, renamed, or withdrawn). **Approvals** (`approvals`): a change submitted for a second person's review, and the decision on it. `approvals.query_change_submitted`, `approvals.query_change_approved`, `approvals.query_change_rejected` (query versions); `approvals.change_submitted`, `approvals.change_approved`, `approvals.change_rejected` (change requests of migration jobs, Custom SQL rules, recipients, data-source connections and the LLM connection, with the requests a submission supersedes); `approvals.ai_plan_submitted`, `approvals.ai_plan_approved`, `approvals.ai_plan_rejected` and `approvals.ai_plan_revision_requested` (AI actor plans); `approvals.ai_actor_created` (an AI actor created, with its data source and whether its changes wait for approval); `approvals.ai_actor_think_started` (an on-demand think cycle accepted as a background task, with the task id); `approvals.ai_actor_archived` (an AI actor archived; archiving one already archived changes and records nothing). **Ownership** (`ownership`): `ownership.owner_changed`: the owner of a subscription, data contract or AI actor changed, with the previous and the new owner as `Users.Id`. Setting the owner a resource already has changes nothing and records nothing. **Alert tasks** (`tasks`): `tasks.task_claimed`, `tasks.task_reassigned`, `tasks.task_resolved` and `tasks.task_auto_resolved` (recorded with the `system` actor when the data recovered). **Grounding curation** (`curation`): `curation.pattern_status_changed` (status before and after), `curation.documentation_patch_applied`, `curation.documentation_patch_rejected`, `curation.golden_case_promoted`, `curation.golden_case_edited`, `curation.relationship_created`, `curation.relationship_verified`, `curation.relationship_unverified`, `curation.relationship_edited`, `curation.relationship_deleted`, `curation.grounding_rechecked` (a run of the grounding recheck, with what it checked and refused), `curation.glossary_term_created`, `curation.glossary_term_updated`, `curation.glossary_term_deleted`, `curation.documentation_section_edited`, `curation.documentation_generation_started` (a documentation regeneration someone started, with its task id) and `curation.documentation_generated` (a documentation generation finished, on the project, by the user who started it or the `system` actor: the new documentation's id, its section count, how many curated sections it kept and how many sections failed to generate). **Data quality** (`data_quality`): `data_quality.contract_created`, `data_quality.contract_updated`, `data_quality.contract_archived` and `data_quality.contract_evaluated` (an evaluation someone started; scheduled evaluations are not recorded). **Data migrations** (`migrations`): `migrations.job_created`, `migrations.job_deleted`, `migrations.job_run_requested` (a run started as a background task, with its task id and where the caller's MigrationOperator permission came from: `role` or `idp`), `migrations.job_started` (a run began (by the caller, or the `system` actor for a scheduled run), recorded before it touches a data source, with the execution id and the source and destination data sources) and `migrations.job_run_ended` (`success`, `failure` with the row counts, or `refused` before it started, with the reason: `not_approved`, `write_target_refused`, `run_in_progress`, or for a background run task, with its task id, `forbidden` or `abandoned`). **Embedding** (`embed`): `embed.grant_created` (with the project, query and version ids, the number of origins and locked parameters and the days until it expires; never an origin or a parameter value) and `embed.grant_revoked`. Revoking a revoked grant records nothing. `embed.token_issued` (a grant secret exchanged for a token, with its lifetime in seconds), `embed.token_refused` (outcome `refused`, the `embed` actor never set: an embed credential that was presented and refused, with the reason `expired`, `invalid_signature`, `revoked`, `grant_expired`, `superseded`, `unknown` or `origin`, and the grant when the credential named one; a request from another origin that carries no credential is refused without an event; at most 600 a minute are recorded, and a client past its refusal budget is answered 429 without one) and `embed.query_rendered` (each render of a grant's query, with the query and version ids, the row count and whether the rows were cut at the cap; outcome `failure` with the reason `render_failed` (also when the run throws), `busy` (no render slot free; within the same 600-a-minute cap as refusals) or `unrunnable`, or `refused` with `parameters`). None carries a token, a secret, an origin, a parameter value or SQL. **The audit itself** (`audit`): `audit.read` (every read of the trail, with the filters' shape and the page), `audit.verified` (every verification, with its counts; outcome `failure` when an event failed, `empty` when nothing was checked; the scheduled verification's also with its checkpoint id and the ids it passed as never committed), `audit.purged` (every retention purge, with the number of events it verified, deleted and left for the next purge; `refused`, deleting nothing, when an event it would delete fails verification) and `audit.mcp_audit_read` (every read of the [MCP audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/#reading-the-audit) export, with the filters' shape and the page), and, by the system, the verification and sealing start of each sealed audit table: `audit.mcp_audit_verified` and `audit.mcp_audit_sealing_started` ([The MCP audit rows](#the-mcp-audit-rows)), `audit.execution_log_verified`, `audit.execution_log_sealing_started`, `audit.setting_history_verified` and `audit.setting_history_sealing_started` ([The execution log and the settings history](#the-execution-log-and-the-settings-history)). ## How events are written - **With the change they record.** An event that records a change is added to the same unit of work as the change (`ISecurityAuditRecorder.StageAsync`, or `SaveWithEventsAsync` when the event needs what the save produces, such as a new id): it commits with the change and rolls back with it, so a rolled-back change leaves no event and an event that cannot be stored fails its change. The actor is resolved before the change runs, so no second connection is opened while the change holds its locks. - **Retries.** `SaveWithEventsAsync` opens its transaction through the context's execution strategy (unless the caller's transaction is open, whose own strategy then owns retries). A bulk update or delete is run again on a retried attempt, on a cleared change tracker, so no event of the failed attempt is saved twice. Tracked changes are saved again only when the failed attempt never saved them; once a save accepted them, the retry fails with nothing committed, and the request fails rather than saving a change without its event. - **On their own.** An event without a change to join (a sign-in, a refusal) is written by `ISecurityAuditRecorder.RecordAsync`, which never fails the action it records. It is not cancelled with the request (an aborted sign-in attempt is still recorded) but is given up after five seconds, so a database that does not answer never holds up a sign-in. An event that could not be written goes to the log stream instead, with every field it would have been stored with (`Beacon.Audit` 9107 at Error, `Persisted=False`), and is counted on the `beacon.security_audit.write_failures` counter (tagged `beacon.security_audit.action`). Alert on that counter: each increment is an event the database does not hold. - **Reads of audit data fail closed.** A read of the trail, of the MCP audit export or of a user's owned resources, and a verification, is recorded before anything is returned, and fails when its event cannot be written: audit data is never handed out without a record of who read it. An identical read (the same filters and page) by the same actor within 60 seconds is recorded once per Beacon process, so paging back and forth does not flood the trail. - **Append-only, inside Beacon.** Beacon refuses to change or delete an event through its data context (an `InvalidOperationException`, before any statement is sent); only the retention purge deletes events. See [What the trail does and does not prove](#what-the-trail-does-and-does-not-prove) for what lies outside Beacon. - **The actor** is the request's caller as Beacon resolved it (`IBeaconActorAccessor`), never a value from the request body; sign-in events name the user they signed in. A host that wires Beacon's sign-in endpoints without `AddBeaconServices` must register `ISecurityAuditRecorder` itself: the sign-in events require it, and a sign-in without it fails instead of going unrecorded. ## Reading the trail The **Security Audit** page (in the Admin section of the sidebar, shown only to an Auditor or an Admin) lists the events newest first, 50 to a page, for a window of whole UTC days: the last seven by default, at most 93. Filter by category, action (the actions of the chosen category), actor user id, resource type and id, and project id; click an actor to show only their events. **Details** shows every field of an event and its details, a withheld value as *withheld*. The page reads nothing while the window is invalid. The same read is `GET /beacon/api/security-audit` (Auditor; any other caller gets `403`): | Query parameter | Required | Notes | |---|---|---| | `from`, `to` | Yes | UTC (a value without an offset is taken as UTC). `to` must not be earlier than `from`, and the window must not exceed 93 days (`400` otherwise). | | `category`, `action`, `actorUserId`, `resourceType`, `resourceId`, `projectId` | No | Exact-match filters. | | `page`, `pageSize`, `sort` | No | Zero-based page, page size up to 200 (default 20), newest first unless `sort` says otherwise. | Every read is recorded first (`audit.read`, and `Beacon.Audit` 9105 on the log stream with the count returned), with which filters were used (a filter's text value only when it is a code) and fails when it cannot be. The trail is not filtered by project membership: an Auditor reads every project's events, which hold no project data. ## Integrity: signatures and verification Each event carries an HMAC-SHA256 over every other field but its database id (in a fixed order, with its time to the microsecond and the id of the key) under the current integrity key, and the id of that key (`KeyId`, `k1-` and a digest that reveals nothing of the key). The database id is not covered: it is generated by the database after the event is sealed. **Verify** checks events against their signatures: - On the Security Audit page, **Verify shown events** checks the events of the current page and **Verify** on a row (or in its details) checks one. Each row then shows the verdict the server gave it: *Intact*, *Modified* (changed or forged), *Unknown key* (signed with a key this deployment does not hold), *Not found* (not in the window), or *Unverified* when the answer named no verdict for it; only *Intact* says the event verified. A banner gives the counts; a verification that found nothing to check says *Nothing was verified* (it is not a success), and while another verification runs the page says *Verification busy*. - `POST /beacon/api/security-audit/verify` with `{ "from": "…", "to": "…" }` checks every event of a window (at most 93 days), and with `"eventIds": [ … ]` (at most 200) only those events of it. Events are read in batches of 1,000. The answer: | Field | Meaning | |---|---| | `checkedCount`, `intactCount` | Events checked, and how many of them verified. | | `modifiedCount` | Events whose fields no longer match their signature: changed or forged, or written without the key. | | `unknownKeyCount` | Events signed with a key this deployment does not hold (a key no longer configured, or a forged key id). It may be genuine; Beacon cannot tell. | | `outOfRangeCount` | Events dated outside the range a genuine event can have (later than five minutes past now, or older than the retention plus two days, taking the longest retention accepted while none is set), counted as failing and not checked. | | `outcome` | `success` (every checked event intact), `failure` (any event failed or was out of range) or `empty` (nothing was checked, which is not a success). | | `failedEventIds` | The ids of the first 100 failing events; the counts say how many failed. | | `verdicts` | For a verification by id: one `{ "eventId", "verdict" }` per requested id, in the order asked: `intact`, `modified`, `unknown_key` or `not_found` (not in the window: never stored, deleted, or out of range). `null` for a whole window. | One verification runs at a time per Beacon process, the scheduled one included; another request answers `429` until it has finished. Every verification is recorded (`audit.verified`, and 9105) before its result is returned, and fails when it cannot be. A verification finds changed and forged events; **it cannot find a deleted one**, which leaves nothing to check; see the checkpoints below. ### Scheduled verification `ISecurityAuditVerificationService.VerifyNewEventsAsync` verifies the events written since its last run, so a changed or forged event is found within the schedule's period without anyone asking: - It continues after the event id its last intact checkpoint names (a `system` `audit.verified` event; one that does not verify itself is passed over and logged) and checks every newer event, in id order, at most 50,000 per run; a backlog is worked off over the following runs. Events are selected by id alone, never by their time: the time is one of the fields the signature covers, so an event whose time was changed fails as *modified*, and one that verifies but is dated where no genuine event can be is counted as out of range. - **The checkpoint advances only as far as every lower id has been checked.** An id with no event below a stored one (an insert whose transaction has not committed yet) is waited for: the checkpoint stays below it, and the events above it, already checked, are checked again by the next run, which also checks the late event once it is there. Once the nearest intact event above the missing id is an hour old, the id is taken as **never committed** (an insert that rolled back still consumes its id) and passed; the checkpoint records how many ids it passed this way and the first and last of them, and the id still waited for, if any. - Events that fail are logged at Error (`Beacon.Audit` 9108, with the counts and the first failing ids) and counted on `beacon.security_audit.verification_failures` (tagged with the action, `audit.verified`). Alert on that counter. - Every run writes a **checkpoint** to the log stream (9109: the event id it started after and its checkpoint id, how many it checked and how many failed, the signature and key id of the checkpoint event, and the ids passed as never committed). Kept in your SIEM, the checkpoints anchor the trail outside the database: an event deleted later leaves a gap below a checkpoint that once covered it, and an id passed as never committed is named there. - A run that checked events records them as a `system` `audit.verified` event, which is the next run's starting point. - Each run then verifies the [MCP audit](#the-mcp-audit-rows) rows written since its own last run, so the one job covers both. The sample host schedules it hourly as the Warp recurring job `security-audit-verify` (`15 * * * *`, `VerifySecurityAuditEventsJob` in `Beacon.SampleProject/Warp/Jobs/McpMaintenanceJobs.cs`). **A host embedding Beacon schedules it itself**, for example with a Warp job of its own like the sample's: ```csharp await recurringJobPublisher.AddOrUpdateRecurringJob(new VerifySecurityAuditEventsJob(), "security-audit-verify", "15 * * * *"); ``` or by calling `ISecurityAuditVerificationService.VerifyNewEventsAsync(cancellationToken)` from its own scheduler. ### Keys and rotation ```json { "Beacon": { "Audit": { "IntegrityKey": "", "PreviousIntegrityKeys": [ "" ], "RetentionDays": null } } } ``` - **`Beacon:Audit:IntegrityKey`** is the secret events are signed with: at least 32 characters, supplied like the encryption key (a secret store or `Beacon__Audit__IntegrityKey`), never committed. A value that is too short or contains `${` stops the host at startup, with a message that never contains the key. Unset, Beacon derives the key, for this purpose only, from `Beacon:Mcp:CallerHashKey`, else from `Beacon:EncryptionKey`; no secret is used as the signing key itself. - **`Beacon:Audit:PreviousIntegrityKeys`** lists keys used earlier: accepted when verifying, never used to sign. Verification also accepts the keys derived from the caller hash key, the encryption key and each `Beacon:PreviousEncryptionKeys` entry, so events signed under the derived key keep verifying while that secret is configured. - **Set `IntegrityKey` before you rotate the encryption key or the caller hash key.** While it is unset, events are signed under a key derived from one of them, and once the old secret leaves the configuration the events signed under it verify as *unknown key*. With a dedicated integrity key, events written from then on do not depend on either. To keep verifying events signed before, keep their secret configured (as the caller hash key, a previous encryption key, or in `PreviousIntegrityKeys`). - **To rotate the integrity key,** set the new value and move the old one to `PreviousIntegrityKeys`. Remove a previous key only once no event you still need to verify was signed with it. - **At startup** Beacon logs, at Information, the id of the key events are sealed under and where it comes from (`integrity_key`, `derived_from_caller_hash_key` or `derived_from_encryption_key`), never the key, and, while `IntegrityKey` is unset, a Warning naming the setting the key is derived from. An event under a key Beacon does not hold is reported as *unknown key*, not as intact: it may be genuine, but Beacon cannot tell. ## Retention Events are kept **forever** by default. Set `Beacon:Audit:RetentionDays` (between 365 and 36,500; any other value stops the host at startup) to delete older events with `ISecurityAuditRetentionService.PurgeExpiredAsync`, which does nothing while `RetentionDays` is unset: - **Verified before deleted.** The purge first verifies every event it is about to delete, in the transaction that deletes them. While any of them fails (changed, or under an unknown key), it deletes nothing: it logs 9108 at Error with the counts and the first failing ids, counts them on `beacon.security_audit.verification_failures` (action `audit.purged`), records a `refused` `audit.purged` event with the failing ids, and throws, so its scheduled job fails visibly. Investigate the events it names before purging: tampered evidence is never deleted unseen. - **Only what it verified is deleted.** Otherwise the purge reads the expired events again, batch by batch, verifies each, and deletes the ones that verify, each by its id together with the signature it verified, never by date. An event changed after it was verified (its signature replaced), one that fails now, and one that became expired while the purge ran (an event whose time was moved back) are not deleted: they are left for the next purge, which verifies them before it deletes anything. - **Deleted with its record.** The `audit.purged` event (by the `system` actor, with the number of events it verified, deleted and left, and the retention) commits with the deletion, in one transaction; `Beacon.Audit` 9106 follows, at Warning when it deleted fewer events than it verified or left expired events behind. - **A purge that does not run** (the database is unreachable, say) is logged at Error (9110, the failure's type and code only), counted on `beacon.security_audit.write_failures` (action `audit.purged`) and rethrown. - **At startup**, a host with a retention set logs that the purge deletes nothing until the host schedules it. The sample host schedules it as the Warp recurring job `security-audit-retention` (`45 3 * * *`, `PurgeExpiredSecurityAuditEventsJob` in `Beacon.SampleProject/Warp/Jobs/McpMaintenanceJobs.cs`). **A host embedding Beacon schedules it itself**, like the [MCP audit purge](https://moberghr.github.io/beacon/mcp/observability-and-audit/#retention), for example with a Warp job of its own like the sample's: ```csharp await recurringJobPublisher.AddOrUpdateRecurringJob(new PurgeExpiredSecurityAuditEventsJob(), "security-audit-retention", "45 3 * * *"); ``` or by calling `ISecurityAuditRetentionService.PurgeExpiredAsync(cancellationToken)` from its own scheduler. ## The execution log Besides the trail, Beacon keeps an **execution log** (`manual_query_execution_logs`) of the statements it runs on a data source for its own features: every statement of a query preview, a query run (from the editor, an AI actor's check, a migration's read), a subscription run and a column value sample, every documentation enum discovery statement and every data contract rule. (MCP calls have their own [audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/).) - **Written before the statement is sent.** Each statement gets a row with `status = Started` (its SQL as sent, without parameter values, the data source and what ran it) before it reaches the data source, and the row is closed when it ends: `Succeeded` (with its row count), `Failed` (with the database's error redacted as it is with PII detection on, whatever the caller's PII setting, so no quoted value is kept; a host data source's generic text as it is), `Cancelled` (the caller stopped it) or `TimedOut` (its statement timeout, on every path), and `completed_time`. A statement the gate refuses never runs and has no row. - **Abandoned rows.** A row still `Started` longer than `Beacon:Query:StatementTimeoutSeconds` plus 10 minutes is a statement whose end was never recorded (the process stopped, the log store failed): the read and the export show it as `Abandoned`, and `status=Abandoned` filters for it (nothing is written; the stored status stays `Started`). - **Who it ran for.** `triggered_by_user_id` is the `Users.Id` the statement ran for: the request's caller, or the owner scheduled work runs as; null for work the system does on its own. Subscription runs record the same id on their run history (`query_execution_history.triggered_by_user_id`). - **Fail-open by default.** A statement whose `Started` row cannot be written still runs, and a Warning is logged. Set `Beacon:Audit:ExecutionLog:Required: true` to refuse such a statement instead (its caller gets an error); with it on, the host fails to start when no execution log (`ISqlExecutionAudit`) is registered. The end of a statement that ran is recorded whatever the caller's cancellation, and never in place of its result. - **Read and export.** `GET /beacon/api/execution-log` (paged, newest first) and `GET /beacon/api/execution-log/export` (CSV, at most 10,000 rows; the `X-Beacon-Truncated` header says when more matched) take `from` and `to` (UTC, at most 93 days apart) and optional `dataSourceId`, `triggeredByUserId`, `status` and `executionContext`. Both are an Auditor's or a SecurityAdmin's, list only the data sources the caller may read, and record an `audit.execution_log_read` or `audit.execution_log_exported` event (the filters' shape, the row count) before they return; they fail when the event cannot be written. - **Values masked on the way out.** The stored SQL keeps the values written into it (bind values are never stored); the read and the export replace its string literals and numbers with `?` and drop its comments, read with the data source's dialect (every quoted span and number when the source or its dialect is unknown). The export also cuts each statement at 4,000 characters (marked ` [cut]`), and writes a cell that starts with `=`, `+`, `-`, `@`, a tab or a line break with a leading `'`, so a spreadsheet reads it as text. - **Retention.** Rows are kept forever unless `Beacon:Audit:ExecutionLog:RetentionDays` is set (greater than zero); schedule `IExecutionLogRetentionService.PurgeExpiredAsync` daily (the sample host's `execution-log-retention` job), which deletes the older rows and logs `Beacon.Audit` event 9111. The log grows with every statement Beacon runs, one row per column value sample of a metadata scan included, so set a retention that fits your storage. - **Not sealed.** The execution log is evidence, not signed like the trail. ## Shipping to a SIEM Every event is logged at category **`Beacon.Audit`** once the transaction that wrote it has committed; an event whose change rolled back is never logged. Point your log shipping at this category (JSON console logs into CloudWatch, a collector, a SIEM agent; see [CloudWatch and collectors](https://moberghr.github.io/beacon/mcp/observability-and-audit/#cloudwatch-and-collectors)) to keep a copy Beacon's database cannot change. | EventId | Name | Level | When | Fields | |---|---|---|---|---| | 9104 | `SecurityAuditRecorded` | Information | An event was committed. | `Action`, `Outcome`, `EventGuid`, `AuditId`, `Persisted` (`True`), `ActorKind`, `ActorUserId`, `ApiKeyId`, `CallerHash`, `ProjectId`, `ResourceType`, `ResourceId`, `Details`, `TraceId` | | 9105 | `SecurityAuditRead` | Information | The trail was read or verified. | The window, which filters were used, the counts | | 9106 | `SecurityAuditPurged` | Information (Warning when it left expired events behind) | The retention purge ran. | Deleted and verified counts, cutoff, retention days, expired events left | | 9107 | `SecurityAuditWriteFailed` | Error | An event written on its own could not be stored. | The fields of 9104 with `Persisted=False` and no `AuditId`, and `Failure` (the failure's type and code) | | 9108 | `SecurityAuditVerificationFailed` | Error | The scheduled verification or the retention purge found stored events that fail verification (or a checkpoint that does). | The counts by verdict, the first failing ids | | 9109 | `SecurityAuditCheckpoint` | Information | Each run of the scheduled verification. | `FromEventId`, `MaxEventId`, `Checked`, `Failed`, `LastHmac`, `KeyId`, `NeverCommitted`, `FirstNeverCommittedId`, `LastNeverCommittedId`, `WaitingForEventId` | | 9110 | `SecurityAuditPurgeFailed` | Error | The retention purge did not run. | The cutoff, the failure's type and code | | 9111 | `McpAuditVerificationFailed` | Error | The scheduled verification found MCP audit rows that fail verification (or an MCP audit checkpoint that does). | The counts by verdict, the first failing row ids | | 9112 | `McpAuditCheckpoint` | Information | Each run of the MCP audit verification. | `FromAuditId`, `MaxAuditId`, `Checked`, `Failed`, `Unsealed`, `LastHmac`, `KeyId`, `SealedFromAuditId`, `NeverCommitted`, `WaitingForAuditId` | | 9113 | `AuditTableVerificationFailed` | Error | The scheduled verification found execution log or settings history rows that fail verification (or a checkpoint of theirs that does). | `Store`, the counts by verdict, the first failing row ids | | 9114 | `AuditTableCheckpoint` | Information | Each run of the verification, once per sealed table. | `Store`, `FromRowId`, `MaxRowId`, `Checked`, `Failed`, `Unsealed`, `LastHmac`, `KeyId`, `SealedFromRowId`, `NeverCommitted`, `WaitingForRowId` | Match a log-stream copy to its row by `EventGuid`. The copy holds the same identifiers, codes and digests as the row, nothing more. | Metric | Type | Alert | |---|---|---| | `beacon.security_audit.write_failures` | Counter, tagged `beacon.security_audit.action` | On any increment: an event written on its own was not stored (its 9107 copy holds it), or a purge did not run (`audit.purged`). | | `beacon.security_audit.verification_failures` | Counter, tagged `beacon.security_audit.action` | On any increment: stored events failed verification (`audit.verified`, `audit.purged`). | **The copy is best-effort.** 9104 is written by the application after the commit, so it is missing when the log category is filtered out or below Information, when the process stops between the commit and the log write, and for a transaction committed directly on the underlying database transaction rather than through Beacon's data context. The database is the source of truth; the copy, the 9107 lines and the checkpoints are what lets you notice when the two disagree. ## What the trail does and does not prove - **Changes and forgeries are found.** An event whose fields were changed, or that was written without the integrity key, fails verification: on request, every hour, and before the purge deletes it. - **Deletions are not found by verification.** Append-only is enforced inside Beacon: its data context refuses to change or delete an event. A database account with write access to the table can still delete rows, and a deleted row leaves nothing to verify. The checkpoints (9109) and the 9104 copies in your SIEM are the anchor outside the database. To make the table append-only at the database itself, give Beacon's connection a role that may not update or delete `SecurityAuditEvents` rows other than through the purge, or add triggers; that is the host's to configure. A chain of signatures across events, which would make a deletion visible inside the trail, is not part of this version. - **The execution log and the settings history are sealed too**, and verified the same way as the MCP audit rows: see [The execution log and the settings history](#the-execution-log-and-the-settings-history), which also shows the DDL a DBA can apply to make the audit tables append-only at the database. - **MCP audit rows are sealed too.** See [The MCP audit rows](#the-mcp-audit-rows): a changed row or statement is found by the next scheduled run, and again later by the rolling re-check; a deleted row that was already checked is found by the row count, one not yet checked only through the checkpoints and the 9100 copies. Rows written before this version are reported as unsealed. The security audit trail records who read that export. ## The MCP audit rows The [MCP audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/) rows (`McpAuditLogs`) are sealed the same way as events, with the same keys and rotation: each row carries an HMAC-SHA256 (`Hmac`) under a key derived from the current integrity key for the MCP audit only, the id of that key (`HmacKeyId`), the version of the field list it covers (`SealVersion`) and a guid of its own (`AuditGuid`, unique in the table, so a row cannot be copied into a second one). The seal covers every field of the row but its database id, and every statement the row holds (`McpAuditStatements`): a statement changed, removed, added or moved to another row fails its row. A row is sealed as it is saved, with what the content retention lock leaves of it. Beacon's data context refuses to change or delete a row or a statement, or to add a statement to a row already saved; the MCP audit retention purge deletes whole rows by age. Each run of the scheduled verification (`ISecurityAuditVerificationService.VerifyNewEventsAsync`) then checks the MCP audit rows written since the last intact `audit.mcp_audit_verified` checkpoint, by id, with their statements, the same way: *intact*, *modified*, *unknown key*, or out of range (a sealed row dated in the future, or older than `Beacon:Mcp:Audit:RetentionDays` allows), waiting for a missing id and passing it as never committed after an hour. Rows checked by an earlier run are checked again too, 10,000 per run, continuing where the last run stopped and starting over at the checkpoint, so a row changed after it was checked is found within one cycle (at an hourly schedule, a table of 240,000 rows is re-checked every day). Each run also counts the rows up to its checkpoint (under a retention, those dated at least two days inside it, which the purge cannot reach before the next run) and compares the count the last run recorded: fewer rows means a checked row was deleted, more that one was written below a checkpoint; both are failures. After a pause longer than two days the comparison is skipped once (a Warning). A row deleted before any run checked it, or one in the oldest two days of the retention, is not found inside the database: the 9100 copies and the checkpoints are the anchor. Rows written before this version carry no seal and are reported as **unsealed**, never as modified. When the host first starts with sealing in place, Beacon records a `system` `audit.mcp_audit_sealing_started` event naming the highest row id written before it; rows up to that id (or up to the first intact sealed row, when that is lower) are unsealed. While an upgrade rolls out, a node still on the version before writes rows without a seal: those dated, by their own time, within a day of that start, and below the first sealed row dated after that day, are unsealed too. Any other row without a seal is a row whose seal was removed, and is reported as modified. A row sealed under the next seal version (written by a node already on a later version) is reported at Warning as *unknown version*, never as modified; one that stays after the upgrade is worth a look. Failures are logged at Error (`Beacon.Audit` 9111) and counted on `beacon.security_audit.verification_failures` (tagged `audit.mcp_audit_verified`); every run writes a checkpoint to the log stream (9112); a run that read rows records them as a `system` `audit.mcp_audit_verified` event, sealed in this trail, which is the next run's starting point. At startup Beacon warns when its data context runs without the interceptor that seals (`SecurityAuditInterceptor`, registered by `UsePostgreSql` / `UseSqlServer`), and when rows exist but no MCP audit verification was recorded for two days (the host does not schedule the verification). Beacon does not migrate its schema at startup: while the seal columns are missing it logs an Error, and writes each MCP audit row without its seal rather than lose it (a Warning per row); those rows are reported as unsealed. Apply the migrations before, or with, the upgrade. A user deleted at the database, outside Beacon, sets `UserId` to null on that user's MCP audit rows (the foreign key's `ON DELETE SET NULL`), which changes sealed rows: they are reported as modified. Beacon itself never deletes a user (a removed user is archived); remove users through Beacon, or expect those findings. ## The execution log and the settings history Two more audit tables are sealed the same way as the MCP audit rows, each under a key of its own derived from the current integrity key: the execution log (`ManualQueryExecutionLogs`, the statements Beacon ran on a data source for its own features) and the settings history (`AppSettingHistory`, one row per changed setting of a save, sensitive values as a mask). Each row carries `Hmac`, `HmacKeyId`, `SealVersion` and its own `AuditGuid` (unique in its table); the seal covers every field but the database id. A row is sealed as it is saved, and Beacon's data context refuses to change or delete one. The same scheduled verification checks both tables after the MCP audit, by id, with the rolling re-check and the row count described above, each from its own checkpoint (`audit.execution_log_verified`, `audit.setting_history_verified`, system events in this trail). Rows written before this version are **unsealed**: at or below the row the once-recorded `audit.execution_log_sealing_started` / `audit.setting_history_sealing_started` event names, or within a day of that start while an upgrade rolls out; any other row without a seal is reported as modified. Failures are logged at Error (`Beacon.Audit` 9113, naming the table) and counted on `beacon.security_audit.verification_failures` (tagged with the table's verified action); every run writes a checkpoint per table to the log stream (9114). The execution log's retention is the shorter of the data retention job's `execution-log` window (`Beacon:Retention:Targets:execution-log`, else `Beacon:Retention:Days`) and the execution log purge (`Beacon:Audit:ExecutionLog:RetentionDays`); nothing purges the settings history. A row the purge deletes past that window is never counted, so it never fails the verification; a counted row deleted inside it is reported as missing. An intact row older than the window (the retention job late, failing or not scheduled) is reported at Warning, never as a failure; a sealed row dated in the future is out of range and fails. An execution log row is written as Started before its statement runs and closed when it ends. The close reads the row, verifies its seal and writes the end together with a new seal over the closed row, only while the row is still Started and carries the seal that was verified; a row whose seal does not verify is closed without a new seal, so it keeps failing verification. The content expiry (`Beacon:Retention:ContentDays`) clears an execution log row's statement text and reduces its error to its class in place. A row whose seal verifies is sealed again over what is left, by the same update, which applies only while the row still carries the seal that was verified; the cleared row keeps verifying, and a later change to it is still found. A row whose seal does not verify is cleared without a new seal, so it keeps failing verification; a row without a seal stays unsealed. A row whose seal the node cannot read (a later seal version during an upgrade, or a key it no longer holds) is left whole, logged at Information, until a node that can verify it clears it. During a rolling upgrade from a version without these seals, run the data retention job only on upgraded nodes: an older node clears sealed rows without sealing them again. Beacon does not migrate its schema at startup. While a table's seal columns are missing (read once from the database's catalog and kept; a missing finding is read again every five minutes), Beacon logs one Warning per table and keeps working without the seal: rows are written without it (no row is lost), the settings history reads as before, the verification passes over the table (the job still completes) and the content expiry clears it as an unsealed store. Apply the migration `AddAuditTableSeal` with the upgrade; rows written before it are unsealed. Beacon.Api records each table's sealing start once the host has started, whether or not the host runs MCP. Two kinds of row are accepted without a seal by design: a row at or below the sealing-started row, and one dated within the upgrade grace before a run has seen a row sealed past it. Someone with write access to the database could strip the seal from such a row unnoticed; the append-only measures below close that. A data source deleted at the database, outside Beacon, sets `DataSourceId` to null on its execution log rows, which is reported as modified; Beacon itself only archives data sources. ### Append-only at the database (for the DBA) Inside Beacon these tables are append-only; a database account that may write them can still change or delete rows. The seals find a change, the row count and the checkpoints a deletion of a row already checked. To refuse both at the database itself, give Beacon's connection a role without `UPDATE` and `DELETE` on the audit tables, and grant `DELETE` only to the account a retention purge runs under, if any. Beacon ships no such DDL; for example, on PostgreSQL: ```sql REVOKE UPDATE, DELETE ON beacon.security_audit_events, beacon.mcp_audit_logs, beacon.mcp_audit_statements, beacon.manual_query_execution_logs, beacon.app_setting_history FROM beacon_app; CREATE FUNCTION beacon.refuse_audit_change() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN RAISE EXCEPTION 'audit rows are append-only'; END $$; CREATE TRIGGER app_setting_history_append_only BEFORE UPDATE OR DELETE ON beacon.app_setting_history FOR EACH ROW EXECUTE FUNCTION beacon.refuse_audit_change(); ``` On SQL Server, `DENY UPDATE, DELETE ON [beacon].[AppSettingHistory] TO [beacon_app];` (and the same for the other tables) does the same. A trigger that refuses `DELETE` also refuses a retention purge of that table: add it only where nothing purges the table, or let it allow the purge's account. Changing data sources and users only through Beacon keeps the foreign keys from rewriting sealed rows. ## Table growth The trail grows with use and, by default, is never trimmed. Sign-ins and reads of audit data are the most frequent events; changes are rarer. A row is a few hundred bytes (details stay under 2,000 characters), indexed by time, by actor and time, by resource, and by `EventGuid`. Plan for it like any audit table: watch its size, set `RetentionDays` to what your retention policy requires (at least a year), and keep the long-term copy in your SIEM. ## Who created and changed what Queries, subscriptions, recipients, data contracts and migration jobs also record who **created** them, who **changed them last**, and when (`CreatedByUserId`, `ModifiedByUserId`, `ModifiedTime`; the caller's identity-provider id, like a resource's owner, never a value from the request). An Auditor or a UserAdmin lists the resources a user owns, created or changed last (for handing them over when the user leaves) in the user dialog or with `GET /beacon/api/users/{id}/owned-resources` (see [User Management → Owned resources](https://moberghr.github.io/beacon/security/user-management/#owned-resources)). Each read is recorded first (`identity.owned_resources_read`); a UserAdmin sees the names only of the resources its projects reach, the others by id and kind. ## Upgrading This version records security-relevant actions in an append-only, signed trail an Auditor reads and verifies, verifies new events on a schedule, records who created and last changed queries, subscriptions, recipients, data contracts and migration jobs, records every read of audit data, keeps the statements every audited MCP call ran on a data source, and lets a deployment pin the MCP audit's content on. See [MCP observability and audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/). - **Events start at the upgrade.** Nothing is backfilled: the trail holds the actions taken after the upgrade only. - **The execution log writes a row before each statement.** From `AddExecutionLogStatus` (PostgreSQL `20261024210000`, SQL Server `20261024210001`, additive: nullable `status`, `completed_time` and `triggered_by_user_id` on `manual_query_execution_logs` plus an index on `triggered_by_user_id` and `created_time`, and a nullable `triggered_by_user_id` on `query_execution_history`; nothing is backfilled) every statement Beacon runs for its own features is in the [execution log](#the-execution-log), subscription steps, column value samples, enum discovery and data contract rules included, so the table grows faster. Older rows keep a null `status`. A step of a query run is recorded as `QueryStep` (the per-step `QueryStepPreview` rows written after the step are gone; a step preview's own statements are `QueryStepPreview`). Earlier actions are only in the logs you kept (the project-access events 7101 onwards, the `Beacon.Audit` events 9100 onwards), which are written as before. - **Existing resources have no attribution.** Queries, subscriptions, recipients, data contracts and migration jobs that exist at the upgrade record no creator, no last modifier and no modification time until they are next changed; a user's owned resources find them by owner only. - **Set `Beacon:Audit:IntegrityKey`, before you change the encryption key or the caller hash key.** Without it, events are signed with a key derived from `Beacon:Mcp:CallerHashKey`, else from `Beacon:EncryptionKey`; once the old secret leaves the configuration, events signed under it verify as *unknown key*. Supply it as a secret (at least 32 characters); see [Keys and rotation](#keys-and-rotation). The host logs the key's id and source at startup, and a Warning while it is unset. - **Schedule two jobs.** The sample host schedules the Warp jobs `security-audit-verify` (hourly at `:15`; it verifies the events written since its last run and writes a checkpoint to the log stream) and `security-audit-retention` (daily; a no-op until `Beacon:Audit:RetentionDays` is set). A host embedding Beacon schedules `ISecurityAuditVerificationService.VerifyNewEventsAsync` and `ISecurityAuditRetentionService.PurgeExpiredAsync` itself; nothing is verified on a schedule, and nothing is purged, until it does. The verification also checks the MCP audit rows. - **MCP audit rows are sealed.** The migration `AddMcpAuditSeal` adds `AuditGuid`, `SealVersion`, `HmacKeyId` and `Hmac` to `McpAuditLogs` (nullable, no backfill) and a unique index on `AuditGuid`. Rows written before it are reported as unsealed. Upgrade every node within a day: past that, a row without a seal is reported as modified. - **The execution log and the settings history are sealed.** The migration `AddAuditTableSeal` adds the same four columns to `ManualQueryExecutionLogs` and `AppSettingHistory` (nullable, no backfill) and a unique index on each `AuditGuid`. Rows written before it are reported as unsealed. Apply it with the upgrade: until it is, rows are written without their seal and not verified (one Warning per table). - **Retention needs no action now.** Events are kept forever until you set `Beacon:Audit:RetentionDays` (365 to 36,500 days). The purge verifies the events it is about to delete first and, while any fails verification, deletes nothing, records a refused `audit.purged` event and throws (its job fails): investigate the events it names. - **Alert on two counters.** `beacon.security_audit.write_failures` (an event written on its own was not stored, and the log stream holds it as 9107 with `Persisted=False`; or a purge did not run, 9110) and `beacon.security_audit.verification_failures` (stored events failed verification, 9108). Ship the `Beacon.Audit` events 9104 to 9114 to your SIEM; the database stays the source of truth, the 9104 copy is best-effort. - **Reads of audit data fail closed.** `GET /security-audit`, `POST /security-audit/verify`, `GET /mcp/audit` and `GET /users/{id}/owned-resources` are recorded in the trail before they return anything and fail when that record cannot be written (so they fail while the database cannot take writes). An identical read by the same actor within 60 seconds is recorded once. One verification runs at a time per process; another answers `429`. - **Owner changes are made only from the owner read.** `PUT /subscriptions/{id}/owner`, `PUT /data-quality/contracts/{id}/owner` and `PUT /ai-actors/{id}/owner` answer `409` when the owner was changed by someone else (or the resource archived) since it was read; nothing is changed. Setting the owner a resource already has stays a no-op: it changes nothing, records nothing and does not check whether that owner may still do the work. - **A host that wires sign-in without `AddBeaconServices`** must register `ISecurityAuditRecorder`: the sign-in, sign-out and single sign-on events now require it, and fail the request without it instead of going unrecorded. - **Plan for the table's growth.** One row per event, kept forever by default; sign-ins and reads of audit data are the most frequent. - **Where content is retained, the MCP audit now stores the SQL of every statement a call ran.** Besides the question, an `ask` keeps the text of its first execution, its repairs, its self-consistency candidates, its value-grounding probes and its dry runs (one `McpAuditStatement` row each, at most 100 per call; past that, a `truncated` marker row counts the rest), and every audit row keeps a keyed `ContentHash` of its parameters. Review this against your privacy policy: switch content retention off for projects that must not keep it, pin it off with `Beacon:Mcp:ForceNoContentRetention`, or bound it with `Beacon:Mcp:Audit:RetentionDays`. Under no retention, statements keep their keyed `SqlHash` and no text. Audit rows written before the upgrade have no statements and no content hash. - **Audit content can be pinned on.** `Beacon:Mcp:Audit:RequireContent: true` pins `RetainQueryContent` on, globally and for every project; a save that switches it off is refused with `409`, an Admin's too, and the MCP Settings page shows the switch disabled with the reason. A call whose project settings cannot be read keeps its content under the lock (logged at Error). With `Beacon:Mcp:ForceNoContentRetention` on as well, the host does not start. - **New pages for an Auditor.** The **Security Audit** page lists, filters and verifies the trail (shown to an Auditor or an Admin only), marking each verified row with the server's verdict; the user dialog of User Management shows a user's **Owned resources** to an Auditor or a UserAdmin, a UserAdmin seeing the names only of resources its projects reach. - **Sign-in attempts name their user, never their name.** A failed or refused password sign-in of an internal user is recorded with the `Users.Id` the sign-in name resolved to and a keyed hash of the name typed, never the name: `AuthenticationResult.BeaconUserId` is set on such a result for the audit (it never signs anyone in), also when the password was right but must be changed first, and `AuthenticationResult.LocalLoginRefused` marks a password refused only because local login is off. - **More actions are recorded**: data source edits (`data_sources.data_source_updated`) and connection changes (`data_sources.credentials_rotated`, recorded from this version on), saved-query locks and MCP tool exposure (`queries.*`), relationships created, glossary terms, documentation sections, migration job runs (`migrations.job_started`, `migrations.job_run_ended`) and reads of the MCP audit and of owned resources; settings changes carry keyed digests of each value before and after. See the [catalogue](#event-catalogue). **Migration.** `AddSecurityAuditTrail` (PostgreSQL `20261014130000`, SQL Server `20261014130001`) is additive and changes no stored value: it creates `security_audit_events` (indexed by `created_time`, by `actor_user_id` and `created_time`, by `resource_type` and `resource_id`, and uniquely by `event_guid`; no foreign keys) and `mcp_audit_statements` (a foreign key to `mcp_audit_logs` that deletes a row's statements with it, and an index on it), adds the nullable `mcp_audit_logs.content_hash` (`character varying(64)` / `nvarchar(64)`), and adds the nullable `created_by_user_id`, `modified_by_user_id` (`character varying(200)` / `nvarchar(200)`) and `modified_time` to `queries`, `subscriptions`, `recipients`, `data_contracts` and `migration_jobs` (PascalCase names on SQL Server). Nothing is backfilled. Rolling it back drops the tables and columns, and with them the trail. Append-only is enforced inside Beacon; to enforce it at the database as well, give Beacon's connection a role that cannot update or delete `security_audit_events` rows outside the purge, or add triggers. **Public API.** - New types and members: - `Beacon.Core.SecurityAudit`: `ISecurityAuditRecorder` (`StageAsync`, `SaveWithEventsAsync` (two overloads), `RecordAsync`, `CallerHashOf`; registered with `TryAdd`, so a host may replace it), `SecurityAuditEntry` (`On`, `InProject`, `With`, `Ending`, `By`, `Naming`), `SecurityAuditDetails`, `SecurityAuditActor` (`Anonymous`, `System`, `OfUser`, `OfCaller`), `SecurityAuditActions`, `SecurityAuditCategories` (with `Queries`), `SecurityAuditOutcomes` (with `Empty`), `SecurityAuditResources` (with `GlossaryTerm`, `DocumentationSection`, `McpAudit`; stored events from earlier versions that name the `dashboard` resource type remain listed), `SecurityAuditActorKinds`, `SecurityAuditIntegrity` (`CurrentKeyId`, `KeySource`, `Seal`, `Verify`, `Digest`, `StoredTime`), `SecurityAuditKeySources`, `SecurityAuditVerdict`, `SecurityAuditVerdicts` (`intact`, `modified`, `unknown_key`, `not_found`), `SecurityAuditLogEvents` (9104–9110: `Recorded`, `Read`, `Purged`, `WriteFailed`, `VerificationFailed`, `Checkpoint`, `PurgeFailed`), `ISecurityAuditRetentionService`, `ISecurityAuditVerificationService` and `SecurityAuditCheckpoint`, `SecurityAuditUserIds` and `Attribution` (`RecordChangeBy`, for a `BeaconActor` or the caller's id). - `SecurityAuditOptions` (`Beacon:Audit`: `IntegrityKey`, `PreviousIntegrityKeys`, `RetentionDays`) and `McpAuditOptions.RequireContent`; `BeaconTelemetry.SecurityAuditWriteFailures` (`beacon.security_audit.write_failures`) and `BeaconTelemetry.SecurityAuditVerificationFailures` (`beacon.security_audit.verification_failures`). - Entities: `SecurityAuditEvent`, `McpAuditStatement`, `IAttributedEntity` (implemented by `Query`, `Subscription`, `Recipient`, `DataContract` and `MigrationJob`, which gain `CreatedByUserId`, `ModifiedByUserId` and `ModifiedTime`), `McpAuditLog.ContentHash` and `Statements`; `BeaconContext.SecurityAuditEvents` and `McpAuditStatements`; `SecurityAuditInterceptor` (`Beacon.Core.Data.Interceptors`, added by both database providers beside the content-retention interceptor). - `Beacon.Core.Mcp`: `IMcpStatementLog` (scoped), `McpStatement`, `McpStatementsTaken` (`Dropped`), `McpStatementPurposes` (with `Truncated`), `McpStatementTables` (`Unparsed`, `More`, `IsMarker`), `McpStatementLogExtensions` and `McpContentHasher`. - Requests: `GetSecurityAuditEventsQuery` and `SecurityAuditEventItem`; `VerifySecurityAuditCommand(From, To, EventIds)` and `VerifySecurityAuditResult` (with `OutOfRangeCount`, `Outcome` and `Verdicts`, a list of `SecurityAuditEventVerdict(EventId, Verdict)`); `GetUserOwnedResourcesQuery`, `OwnedResourceItem` (with `NameWithheld`) and `OwnedResourceKind`; `McpAuditStatementItem`; `McpDeploymentLock` and `GetMcpDeploymentLocksResult.Locks`. - `AuthenticationResult.LocalLoginRefused`. - The sample host's Warp jobs `PurgeExpiredSecurityAuditEventsJob` and `VerifySecurityAuditEventsJob`, with their handlers (the sample is not shipped; copy them, or schedule the services another way). - Changed signatures (code that constructs or implements these must change): - `TaskService` and `AppSettingsService` take an `ISecurityAuditRecorder` (registered by `AddBeaconServices`); a host that builds either itself passes one. - `McpAuditLogItem` (a positional record) gains `ContentHash` and `Statements`. - `IMcpStatementLog.TakeAll()` returns `McpStatementsTaken`, an `IReadOnlyList` that also counts the statements not kept: code that reads the list compiles unchanged; an implementation of the interface changes. - Changed behaviour: - The changes the trail records stage their event in the same unit of work, so an event that cannot be stored fails that change; events written on their own (sign-ins, refusals) never fail their action and give up after five seconds, logging the event (9107) instead. Reads of audit data fail when their event cannot be written. - `ISecurityAuditRecorder.SaveWithEventsAsync` replays a retried attempt only when the failed attempt never saved its tracked changes; otherwise the retry fails with nothing committed. - `ISecurityAuditRetentionService.PurgeExpiredAsync` throws `InvalidOperationException` when it refuses (an expired event fails verification), and rethrows when it cannot run. - The sign-in events resolve `ISecurityAuditRecorder` as a required service. - `AuthenticationResult.BeaconUserId` is also set on a failed password check, and on a right password that must be changed first, as above; saving MCP settings that contradict `Audit:RequireContent` throws `SettingLockedException` (`409`). - Owner changes of subscriptions, data contracts and AI actors throw `BeaconConflictException` (`409`) when the owner changed since it was read. - `McpContentHasher.Hash(null)` differs from `Hash("")`. - REST and OpenAPI: - New routes: `GET /beacon/api/security-audit` (`from`, `to`, `category`, `action`, `actorUserId`, `resourceType`, `resourceId`, `projectId`, `page`, `pageSize`, `sort`; a paged list of `SecurityAuditEventItem`) and `POST /beacon/api/security-audit/verify` (`from`, `to`, optional `eventIds`; a `VerifySecurityAuditResult` with `checkedCount`, `intactCount`, `modifiedCount`, `unknownKeyCount`, `outOfRangeCount`, `outcome`, `failedEventIds` and `verdicts`; `429` while another verification runs), both for an Auditor; `GET /beacon/api/users/{id}/owned-resources` (`kind` as a number, `page`, `pageSize`, `sort`; a paged list of `OwnedResourceItem`, with `nameWithheld`), for an Auditor or a UserAdmin. - New schema `SecurityAuditEventVerdict` (`eventId`, `verdict`). - New response fields: `locks` on `GET /beacon/api/mcp/settings/locks`; `contentHash` and `statements` on each row of `GET /beacon/api/mcp/audit`. - New `409`s: `PUT /beacon/api/mcp/settings` and `PUT /beacon/api/mcp/projects/{projectId}/settings` that switch content retention off under `Audit:RequireContent`; `PUT /beacon/api/subscriptions/{id}/owner`, `PUT /beacon/api/data-quality/contracts/{id}/owner` and `PUT /beacon/api/ai-actors/{id}/owner` when the owner changed meanwhile. - `GET /beacon/api/mcp/audit` and the security audit and owned-resources routes answer an error, and return nothing, when their read cannot be recorded. --- # Admin Settings > The Admin settings page: the base URL and the LLM provider, changed at runtime and swapped without a restart, who may read or change each setting, and the settings change history. Source: https://moberghr.github.io/beacon/security/admin-settings/ **Admin settings** (`/beacon/admin-settings`) holds the base URL Beacon puts in outgoing links and the LLM provider connection. A save applies to the running process at once, without a restart. Every change is recorded in a change history that also covers MCP settings and migration-write-target changes. ## Who may do what The page is open to Admins and to two [functional permissions](https://moberghr.github.io/beacon/security/authorization/#functional-permissions): an **Auditor** reads it, a **SecurityAdmin** reads and changes it. The LLM connection, the LLM concurrency and per-minute limits, and the base URL stay an Admin's. | Action | Route | Admin | SecurityAdmin | Auditor | |---|---|---|---|---| | Read the settings (secrets shown only as set or not set) and the change history | `GET /beacon/api/admin-settings` | ✓ | ✓ | ✓ | | List the LLM providers this host installed | `GET /beacon/api/admin-settings/llm-providers` | ✓ | ✓ | ✓ | | Change the models and the monthly budget | `PUT /beacon/api/admin-settings` | ✓ | ✓ | – | | Change **Max concurrent requests**, **Tokens per minute**, **Requests per minute** or **Enforce requests/tokens per minute** | `PUT /beacon/api/admin-settings` | ✓ | – (`403`) | – | | Change the **Base URL** | `PUT /beacon/api/admin-settings` | ✓ | – (`403`) | – | | Change the LLM **provider**, **endpoint**, **region**, Bedrock **authentication mode** or any **credential** | `PUT /beacon/api/admin-settings` | ✓ | – (`403`) | – | | Test the stored LLM connection | `POST /beacon/api/admin-settings/test-llm` | ✓ | ✓ | – | | Test with another provider, endpoint, region, authentication mode or credential | `POST /beacon/api/admin-settings/test-llm` | ✓ | – (`403`) | – | The provider, endpoint, region and authentication mode decide where Beacon sends its LLM requests with the stored credentials, the base URL is the address of every link Beacon sends out, and the concurrency and per-minute limits decide how much LLM traffic runs at all, so only an Admin changes them. For anyone else the page shows those fields disabled with a note, and a request that changes any of them answers `403` and saves nothing. An Auditor sees the page read-only. API keys never reach these routes. While the [approval workflow](https://moberghr.github.io/beacon/security/approvals/#migration-jobs-custom-sql-rules-and-recipients) covers the LLM connection (`CoversLlmConnection`), an Admin's change of the provider, endpoint, region or authentication mode, or a first stored provider, is not applied on save: it waits for a second live Admin's approval, and the stored connection and models stay in use until then. The other values of the same save apply at once; a change of credentials or models alone, and testing the connection, are not reviewed. ## The page ### General | Field | What it does | |---|---| | **Base URL** | The address Beacon puts in notification links (`{BaseUrl}/notifications/{id}` in Teams, Slack and webhook messages). The UI is served under `/beacon`, so include it: `https://beacon.example.com/beacon`. A saved value is an absolute `https` URL without user information or a fragment (plain `http` only for `localhost` or a loopback address while the host runs in Development); anything else answers `400`. A stored value that is not usable leaves the link out of messages. | ### AI provider Pick the provider; the page shows only the fields it needs. | Provider (UI) | `AiProvider` value (API `llmProvider`, `Beacon:LLM:Provider`) | Fields | |---|---|---| | **OpenAI** | `OpenAI` (0) | API key, Model, Fast model | | **Anthropic** | `Claude` (1) | API key, Model, Fast model | | **Azure OpenAI** | `AzureOpenAI` (2) | Endpoint, API key, Deployment (model), Fast deployment | | **AWS Bedrock** | `Bedrock` (3) | Region, Credentials, Model, Fast model | AWS Bedrock is installed only when the host references `Moberg.Beacon.AI.Bedrock` and calls `AddBeaconBedrock()`. The page offers only the providers the host installed (a stored provider that is not installed shows as **Package not installed**); picking another one is refused with `400`. **Bedrock credentials.** The authentication mode under **Credentials** decides which credentials the client uses. **IAM role** (the default) uses the default AWS credential chain (instance role, environment). **Access key** uses the AWS access key id and secret access key, and **Temporary credentials** uses them with a session token. Both of these modes need both keys: one without the other fails instead of falling back to the default chain. On `PUT /beacon/api/admin-settings` the fields are `llmBedrockAuthMode`, `llmAwsAccessKeyId`, `llmAwsSecretAccessKey` and `llmSessionToken`; in configuration they are `Beacon:LLM:BedrockAuthMode`, `AwsAccessKeyId`, `AwsSecretAccessKey` and `SessionToken` (see [Configuration → LLM](https://moberghr.github.io/beacon/getting-started/configuration/#ai--llm)). The role or keys need the `bedrock:InvokeModel` permission for the chosen model. An install that stored the key pair as `accessKey:secretKey` in the API key field keeps working. With both key fields empty, the access key and temporary credential modes use that pair. The IAM role mode uses it as well, but logs a warning (without the keys) that static keys from the API key field are in use instead of the default chain; clear the API key field to use the IAM role, or move the keys to the access key mode. When a saved configuration cannot build a provider, no provider is active until the configuration is fixed: AI features answer that the LLM provider is not configured, and the previous provider is never kept. The log names the failure's type only. **Test connection** sends a tiny request (at most 16 output tokens) with the stored connection and the models on the form, without saving. Credentials and the Azure endpoint are encrypted at rest with `Beacon:EncryptionKey` and never returned: the page shows only whether each is set. Over the API, `null` for a credential or the endpoint keeps the stored value. The stored API key (or, for Bedrock, the AWS secret access key) is never sent to a different provider or endpoint: a save or a test that changes the provider or endpoint must carry the key again, or it answers `400` and nothing is saved or sent. The REST API sends enums as integers; configuration takes the names. ### Rate limits & budget **Max concurrent requests** (default 50) applies to the LLM request queue as soon as it is saved: a higher limit lets waiting calls run at once, a lower one takes effect as running calls finish (none is cancelled), and background work keeps at most half the slots. **Tokens per minute** and **Requests per minute** (defaults 80,000 and 1,000) apply only while **Enforce requests/tokens per minute** is on; it is off by default, and while it is off the two fields are recorded but not applied. When it is on, each call reserves one request and an estimate of its tokens (prompt characters divided by four, plus the call's maximum answer tokens) before it is sent, and the tokens are settled with the usage the provider reports. The reservation is taken before the call waits for a concurrency slot and held while it waits; a call that is never sent (refused, cancelled or timed out) gives it back. The tokens are settled only when the response reports its input tokens; otherwise the estimate stands. Calls the limits cannot admit at once wait in arrival order, so a large call is never overtaken by smaller ones; an interactive call may pass waiting background work, and background work leaves a quarter of each per-minute allowance for interactive calls. A call that cannot get its share within its queue wait fails with *The LLM is busy. Try again later.* `0` means unlimited for that dimension. **Monthly budget (USD)** (default 100) is recorded, not enforced: Beacon keeps no record of spend, so calls continue past it. Use the provider's own spend limits for a hard cap. Over the API, `llmRateLimitsEnabled` left out or `null` keeps the stored switch. Only an Admin changes these four values (`403` otherwise): a SecurityAdmin sees them and sends the displayed ones back, and a SecurityAdmin's save neither stores nor applies them, so the host-configured limits, or the ones an Admin saved, stay in force. Once an Admin saves them, the stored limits replace the host-configured ones (`Beacon:LLM:Limits`), also after a restart, and the first time they differ from the configured ones the host logs it at Information (numbers only). Saving the settings records each changed value, the enforcement switch included (`LLM.RateLimitsEnabled`), in the change history and the security audit. ### Recent changes The page lists the latest changes from the settings history: when, which key, and who. ## How a save applies 1. The page sends `PUT /beacon/api/admin-settings`. 2. `AppSettingsService` writes the values to `app_settings` (sensitive ones encrypted), records each changed key in `app_setting_history`, and clears its cache. 3. The base URL in the running process (`BeaconConfiguration.BaseUrl`) is replaced. 4. `ILlmConfigurationUpdater.UpdateConfiguration` hands the settings to `LlmProviderManager`, which builds a new provider and swaps it in atomically. A save by anyone but an Admin writes no LLM limit and keeps the limits in force. `DelegatingLlmProvider`, the proxy every feature holds, uses the new provider for the next request. **At startup** the LLM provider is built from `Beacon:LLM` and the base URL comes from the host's code (`options.BaseUrl`; the sample host reads `Beacon:BaseUrl`). Saving Admin settings swaps them immediately. After a restart the stored settings replace the configured ones once they are read: a stored base URL, the stored LLM configuration when a provider was stored, and the LLM limits (concurrency, per-minute limits and their enforcement) only when an Admin stored them, also when no provider was stored. A save that stores no provider keeps the provider from `Beacon:LLM` in force, as a restart would: it applies the limits (an Admin's save) and the budget only. See [Configuration → AI / LLM](https://moberghr.github.io/beacon/getting-started/configuration/#ai--llm). Settings are cached in memory for one hour; a save clears the cache. ## Change history `app_setting_history` records one row per changed value: `setting_key`, `old_value`, `new_value` (`***` for sensitive values), `changed_at` and `changed_by_user_id`. It is returned with `GET /beacon/api/admin-settings` (the latest 200 rows), so Auditors, SecurityAdmins and Admins read it. Three families of keys are written: | Keys | Written when | |---|---| | `General.BaseUrl`, `LLM.*` (for example `LLM.Provider`, `LLM.ApiKey`, `LLM.Model`) | Admin settings are saved | | `Mcp.{Setting}`, `Mcp.Project.{projectId}.{Setting}` | an MCP setting changes, globally or for a project | | `DataSource.{id}.AllowMigrationWrites` | an Admin marks or unmarks a data source as a [migration write target](https://moberghr.github.io/beacon/features/data-migration/) | Every save that changes settings is also recorded in the [security audit trail](https://moberghr.github.io/beacon/security/audit-trail/) (`settings.admin_settings_changed`, `settings.mcp_settings_changed`, `settings.mcp_project_settings_changed`), naming the keys that changed and the history rows that hold their values, with a keyed digest of each value before and after; the values themselves stay in the history. Security-relevant actions outside settings (sign-ins, role changes, membership changes and the rest) are recorded in the security audit trail, not as history rows; see [Security model](https://moberghr.github.io/beacon/security/#12-audit-and-security-events). ## Programmatic access ```csharp public sealed class MyService(IAppSettingsService settingsService) { public async Task UseAsync(CancellationToken ct) { var settings = await settingsService.GetSettingsAsync(ct); // BaseUrl, LlmProvider, LlmModel, … settings.LlmModel = ""; await settingsService.SaveSettingsAsync(settings, userId: "", ct); var llmHistory = await settingsService.GetHistoryAsync(key: "LLM.Model", ct); } } ``` `SaveSettingsAsync` applies no permission check: the REST route does. Code that calls it directly is responsible for who may. `ILlmConfigurationUpdater` (`void UpdateConfiguration(AppSettingsData settings)`) lives in `Beacon.Core` so `AppSettingsService` can call it without referencing `Beacon.AI`; `LlmProviderManager` implements it. ## MCP settings MCP behaviour has its own page, **MCP settings** (`/beacon/mcp-settings`): an Auditor or a SecurityAdmin reads it, a SecurityAdmin changes it, Admins do both. Turning read-only enforcement or PII detection off, globally or for a project, is an Admin's alone (`403` otherwise). Its tabs are **Pre-prompt**, **Tool descriptions**, **Guardrails** and **Context preview**; a scope selector switches between the global defaults and a project's overrides. The deployment can pin settings no one can change at runtime, through `Beacon:Mcp`: | Lock | Default | Pins | |---|---|---| | `ForceReadOnly` | `true` | read-only enforcement on, globally and for every project | | `ForcePiiDetection` | `true` | PII detection on | | `ForceNoContentRetention` | `false` | content retention off (questions, SQL and free-text errors never persisted) | | `ForbidRowDataToLlm` | `false` | sample-value collection, value grounding and the eval judge off | On the global settings a locked switch is shown **disabled** with a lock and a note naming the configuration key; in a project's scope locked fields are not offered and a banner names them. A write that contradicts a lock answers `409`, an Admin's included; `GET /beacon/api/mcp/settings/locks` lists the pinned settings. `Beacon:Mcp:Ceilings` clamp numeric settings downward: `MaxRowLimit` caps the row limit the MCP tools apply, `MaxSqlChars` and `MaxQuestionChars` are always-on length limits, `AskTimeoutSeconds` and `AskMaxLlmCalls` are the always-on time and LLM-call budgets of one `ask` call, and `AskSchemaContextChars` bounds the schema context one `ask` sends to the LLM; the other ceilings clamp the setting of the same name shown on this page. Every MCP settings change is recorded in the history above. See [Configuration → Deployment locks](https://moberghr.github.io/beacon/getting-started/configuration/#deployment-locks) and [MCP server](https://moberghr.github.io/beacon/mcp/server/). ## See also - [AI integration](https://moberghr.github.io/beacon/features/ai-integration/): the features that use the LLM connection - [MCP server](https://moberghr.github.io/beacon/mcp/server/): MCP settings and guardrails - [Authorization](https://moberghr.github.io/beacon/security/authorization/#functional-permissions): Auditor and SecurityAdmin - [Configuration](https://moberghr.github.io/beacon/getting-started/configuration/): `Beacon:LLM` and the startup values --- # Configuration > Every configuration key Beacon reads, grouped by area with type, default and validation; the code-only options set in AddBeaconServices; the deployment locks; and the environment-variable form. Source: https://moberghr.github.io/beacon/getting-started/configuration/ This is the reference for everything you can configure in Beacon. It applies whether you run the `Beacon.SampleProject` host or embed the `Beacon.*` packages in your own ASP.NET Core application; [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/) shows the registration calls in context. ## How configuration is read Beacon has two kinds of settings: 1. **Configuration keys**, read from `IConfiguration`: `appsettings*.json`, environment variables, user secrets or a secrets provider. Everything under [Core and database](#core-and-database) through [Telemetry](#telemetry) below. 2. **Code-only options**, set in the `AddBeaconServices(configuration, options => …)` lambda. They are not read from configuration. See [Code-only options](#code-only-options). Rules that apply throughout: - **Validated at startup.** Most sections are validated when the host starts; a bad value stops the host with a message that names the key (never a secret's value). Sections marked "validated at first use" are checked when the feature first reads them. - **Secrets never go in `appsettings.json`.** Supply `Beacon:EncryptionKey`, connection strings, the OIDC client secret and LLM keys through environment variables or a secrets provider. .NET configuration does not expand `${...}` placeholders: the text itself would become the value (for `Beacon:EncryptionKey` and `Beacon:Mcp:CallerHashKey` startup refuses it). - **The sample host.** `src/Beacon.SampleProject/appsettings.json` holds only `Logging`; the sample's Beacon settings are in `appsettings.Development.json`, which loads only with `ASPNETCORE_ENVIRONMENT=Development`. Outside Development, supply every value yourself. ### Environment-variable form Replace each `:` with a double underscore. List items take an index; dictionary keys are path segments. ```bash export ConnectionStrings__BeaconContext="Host=db;Database=beacon;Username=beacon;Password=…" export Beacon__EncryptionKey="" export Beacon__PreviousEncryptionKeys__0="" export Beacon__UserManagement__LocalLogin__BreakGlassUserNames__0="superadmin" export Beacon__Query__MaxExecutionRows=50000 export Beacon__Mcp__ForceReadOnly=true ``` An Azure Key Vault secret named `Beacon--EncryptionKey` maps to `Beacon:EncryptionKey` the same way. ## Host setup in Program.cs A condensed version of the sample host's wiring. The SPA is served at **`/beacon`**, so cookie authentication, the login and setup endpoints and the middlewares take that base path. ```csharp using Beacon.AI; using Beacon.Api; using Beacon.Api.Authentication; using Beacon.Api.Endpoints; using Beacon.Core; using Beacon.Core.PostgreSql; using Beacon.MCP; using Beacon.MCP.Discovery; using Beacon.UI; builder.Services.AddBeaconHostInfrastructure(); builder.Services.AddBeaconServices(builder.Configuration, options => { options.AddBeaconScheduler(); // required: your IBeaconScheduler options.BaseUrl = builder.Configuration["Beacon:BaseUrl"]; options.UseAI = true; options.AddEmailAdapter(); options.Authorization.Enabled = true; options.Authentication.EnableLoginForm = true; options.AddAuthenticationProvider(); options.EnableUserManagement(); }) .AddPostgreSqlConnector() .AddSqlServerConnector() // … other connectors … .UsePostgreSql(); // ConnectionStrings:BeaconContext, schema "beacon" builder.Services.AddBeaconCookieAuthentication(BeaconUiEndpointRouteBuilderExtensions.BasePath); // "/beacon" builder.Services.AddBeaconDataProtection(builder.Configuration, builder.Environment); builder.Services.AddBeaconOidcAuthentication(builder.Configuration); // optional SSO builder.Services.AddBeaconAI(builder.Configuration); builder.Services.AddBeaconMcp(); builder.Services.AddOpenApi(); builder.Services.AddAntiforgery(x => { x.HeaderName = "X-XSRF-TOKEN"; x.Cookie.Name = ".Beacon.Antiforgery"; }); builder.Services.AddBeaconApiAuthorization(); builder.Services.AddBeaconApiServices(); var app = builder.Build(); var beaconConfiguration = app.Services.GetRequiredService(); app.UseStaticFiles(); app.UseMcpResourceMetadataChallenge(); app.UseBeaconAuthentication(); // API key → bearer JWT → UseAuthentication → Beacon.Auth cookie app.UseAuthorization(); app.UseMiddleware(beaconConfiguration, BeaconUiEndpointRouteBuilderExtensions.BasePath); // with the login form app.UseMiddleware(beaconConfiguration, BeaconUiEndpointRouteBuilderExtensions.BasePath); // with user management app.UseAntiforgery(); app.MapBeaconOpenApi(); // /openapi/v1.json app.MapBeaconApi(); // /beacon/api/* and the hub /beacon/api/hub app.MapLoginEndpoints("/beacon", beaconConfiguration); // /beacon/api/auth/login, logout, sso app.MapSetupEndpoints("/beacon"); // /beacon/api/setup/* (with user management) app.MapBeaconMcp(); // /beacon/mcp, Execute-scope policy app.MapMcpDiscovery(); // /.well-known/* (anonymous) app.MapBeaconUi(); // the React SPA at /beacon ``` The middleware order is load-bearing; keep it. `UseBeaconAuthentication()` takes the place of `UseAuthentication()` and wires Beacon's half of it; the Beacon middlewares act on `/beacon` and [`Beacon:Authentication:AdditionalPaths`](#where-beacon-credentials-apply) only. Beacon does **not** create or migrate its own schema when the host starts: apply its migrations as described in [Metadata database](https://moberghr.github.io/beacon/integrate/database/). | Path | What | |---|---| | `/beacon` | the React UI (`/beacon/login`, `/beacon/setup`, …) | | `/beacon/api/*` | REST API; `/beacon/api/health` is anonymous | | `/beacon/api/hub` | SignalR hub | | `/beacon/mcp` | MCP server | | `/.well-known/oauth-protected-resource`, `/.well-known/mcp/server-card.json` | MCP discovery (anonymous) | | `/openapi/v1.json` | OpenAPI document (open in Development; signed-in callers only elsewhere unless `Beacon:OpenApi:Anonymous` is `true`) | | `/warp` | Warp dashboard (sample host, Admin only; listed in `Beacon:Authentication:AdditionalPaths`) | ## Code-only options Set in the `AddBeaconServices` lambda. None of these are read from configuration, except the two noted. | Option | Default | What it does | |---|---|---| | `AddBeaconScheduler()` | none, **required** | Your `IBeaconScheduler`; startup fails without one (see [Scheduling](#scheduling)) | | `AddEmailAdapter()` | none | Your `IEmailAdapter`; Beacon ships no email implementation (see [Email adapter](#email-adapter)) | | `BaseUrl` | `null` | The address put in notification links: `{BaseUrl}/notifications/{id}`. Include the `/beacon` path (`https://beacon.example.com/beacon`). Not read from configuration by Beacon; the sample host reads `Beacon:BaseUrl` into it. Admin settings can replace it at runtime (see [Base URL](#base-url-configuration)) | | `UseAI` | `false` | Turns on the AI features; with `false` they are registered as stand-ins that report AI as unavailable | | `Authorization.Enabled` | `false` | Enforces global roles on `/beacon/api` (see [Authorization](https://moberghr.github.io/beacon/security/authorization/)) | | `AddAuthorizationProvider()` | `DefaultAuthorizationProvider` (allows all); `DatabaseAuthorizationProvider` with user management | The global-permission provider | | `Authentication.EnableLoginForm` | `false` | The login form and the login redirect | | `AddAuthenticationProvider()` | none | How a login-form password is checked (`DatabaseAuthenticationProvider`, `JwtExternalApiAuthenticationProvider`, `HybridAuthenticationProvider` or your own) | | `Authentication.LoginRedirectPath` | `/beacon/home` | Where a sign-in lands (a full browser path) | | `Authentication.LoginPath` | `/login` | The login page, under the base path | | `Authentication.CookieExpirationHours` | `24` | Requested session-cookie lifetime; capped by `Beacon:Authentication:Session:AbsoluteLifetimeHours` (default 12) | | `Authentication.RememberMeExpirationDays` | `30` | Requested "remember me" lifetime; capped the same way | | `UserManagement.Enabled` (`EnableUserManagement()`) | `false` | Beacon's user store, Users page and first-run setup | | `UserManagement.AllowInternalUsers` | `true` | Allow password users | | `UserManagement.MinimumPasswordLength` | `8` | Shortest password | | `UserManagement.RequirePasswordComplexity` | `true` | Upper case, lower case, digit and another character required | | `UserManagement.SetupToken` | unset | Also bound from `Beacon:UserManagement:SetupToken` (see [User management](#user-management-and-first-run-setup)) | | `UserManagement.LocalLogin` | on | Also bound from `Beacon:UserManagement:LocalLogin` | | `ApprovalWorkflow.*` (`EnableApprovalWorkflow(…)`) | off | The four-eyes workflow (see [Approval workflow](#approval-workflow)) | | `SavedQueryTools.NamedToolLimit` | `25` | Up to this many saved-query MCP tools per caller are listed as their own `q_` tools; above it, callers get `search_saved_queries` and `run_saved_query`. Must not be negative | Bearer-token and external-login settings are code-only too, through `builder.Services.AddBeaconJwtAuthentication(jwt => …)` (see [JWT bearer](#jwt-bearer)). ## Core and database | Key | Type | Default | What it does | |---|---|---|---| | `ConnectionStrings:BeaconContext` | string | **required** | Beacon's own database (PostgreSQL or SQL Server), read by `.UsePostgreSql()` / `.UseSqlServer()` without arguments. Missing: startup fails | | `ConnectionStrings:WarpContext` | string | falls back to `BeaconContext` | Sample host: the Moberg.Warp job store, PostgreSQL only, `warp` schema. When Beacon runs on SQL Server, point it at a PostgreSQL database | | `ConnectionStrings:{Name}` | string | none | The connection of a host-managed data source, which stores only the name and reads the string at call time | | `Beacon:Schema` | string | `beacon` | **Not configurable.** Any value other than `beacon` stops the host (see below). Leave it out | | `Beacon:BaseUrl` | string | none (the sample host falls back to `https://localhost:7187`) | Read by the sample host into `options.BaseUrl` | | `Beacon:PublicBaseUrl` | string (origin) | derived from the request | The origin advertised in the MCP discovery documents and the `WWW-Authenticate` challenge. Unset, it follows the request's `Host` header, which the client controls, the documents are sent with `Cache-Control: no-store`, and startup logs a Warning. Set it behind any proxy | | `Beacon:OpenApi:Anonymous` | bool | `false` | Publishes the OpenAPI document (`MapBeaconOpenApi()`) to anonymous callers outside Development. Off, it needs a signed-in Beacon caller; list `/openapi` in `Beacon:Authentication:AdditionalPaths` so the Beacon session reaches it. | | `Beacon:ForwardedHeaders:Enabled` | bool | `false` | Sample host: apply `X-Forwarded-For` and `X-Forwarded-Proto` from known proxies | | `Beacon:ForwardedHeaders:KnownProxies` | string[] | empty | Sample host: proxy IP addresses to trust; entries that are not IP addresses are skipped, and an empty list trusts none | | `AllowedHosts` | string | None | Standard ASP.NET Core host filtering | | `Logging:*` | n/a | n/a | Standard logging. For JSON console logs with trace ids: `Logging:Console:FormatterName=json`, `Logging:Console:FormatterOptions:IncludeScopes=true` | ### Schema configuration Beacon runs in the **`beacon`** schema and only that schema. Its migration snapshots bake the schema in, so any other schema would make `Database.Migrate()` fail with `PendingModelChangesWarning`; Beacon refuses the setting up front: ``` Beacon only supports the 'beacon' schema, but 'tenant_a' was configured. Remove Beacon:Schema (or set it to 'beacon') and move any existing Beacon tables into 'beacon'. ``` The obsolete overloads `.UsePostgreSql(connectionString, schema)` and `.UseSqlServer(connectionString, schema)` (removed in 6.0) accept a schema argument for the same reason only as `beacon`, and otherwise ignore it. On PostgreSQL the search path is `beacon,public`, so a pgvector extension installed in `public` resolves. Startup logs a Warning when every role can create objects in `public` (the default before PostgreSQL 15); run `REVOKE CREATE ON SCHEMA public FROM PUBLIC` unless every role that can connect is trusted. ### Forwarded headers The forwarded-headers keys are read by the **sample host** only. A host embedding the packages calls `UseForwardedHeaders` itself. Behind a proxy this matters: the login and setup throttles key on the client address, and `SameAsRequest` cookie security keys on the scheme. Without forwarded headers the cookie is issued without the `Secure` flag behind a TLS-terminating proxy: honour the forwarded scheme, or set `Beacon:Authentication:Cookie:SecureAlways` to `true` (default `false`; the cookie is then never sent over plain HTTP, so sign-in needs HTTPS end to end or a proxy that terminates it). Outside Development, Beacon logs a Warning at startup when the server listens on a plain HTTP address and the option is off. Add HTTPS redirection and HSTS in the host. ```json { "Beacon": { "ForwardedHeaders": { "Enabled": true, "KnownProxies": [ "10.0.0.5" ] } } } ``` ### Base URL configuration `BaseUrl` is the address of the Beacon UI in outgoing links (Teams, Slack, webhooks: `{BaseUrl}/notifications/{id}`). The UI lives under `/beacon`, so include it: ```json { "Beacon": { "BaseUrl": "https://beacon.example.com/beacon" } } ``` At startup the base URL is what the host's code sets (`options.BaseUrl`). Saving **Base URL** in [Admin settings](https://moberghr.github.io/beacon/security/admin-settings/) replaces it in the running process at once; after a restart the startup value applies again until the settings are saved. Keep the two in step. ## Encryption key configuration (required) | Key | Type | Default | What it does | |---|---|---|---| | `Beacon:EncryptionKey` | string (base64) | **required** | Encrypts data-source connection data, repository tokens, sensitive settings and recipient destinations at rest (AES-256-GCM, per-value nonce; the AES key is the SHA-256 of the key text). Checked in every environment | | `Beacon:PreviousEncryptionKeys` | string[] | empty | Older keys, used only to decrypt | | `Beacon:Mcp:CallerHashKey` | string | derived from the encryption key | Secret behind the MCP caller hashes in the audit; keeps them stable across a key change | Generate a key with: ```bash openssl rand -base64 32 ``` The key must be **base64 that decodes to at least 32 random bytes**: a passphrase, a short value, base64 of text (every decoded byte printable) or of a repeated pattern (fewer than 16 distinct byte values) is refused. A published sample value and a value containing `${` are refused too. The message never contains the key. ```bash # Development dotnet user-secrets --project src/Beacon.SampleProject set "Beacon:EncryptionKey" "$(openssl rand -base64 32)" # Elsewhere export Beacon__EncryptionKey="" ``` API keys are not encrypted: they are SHA-256-hashed, and the raw key is shown once. ### Replacing the key Values are always written with `Beacon:EncryptionKey`. To replace a key, set the new one and list the old one in `Beacon:PreviousEncryptionKeys` (as a list: `Beacon__PreviousEncryptionKeys__0`; a single value is read as one key, with a startup Warning). Previous keys only decrypt and are not strength-checked, since their purpose is to retire a weak key. ```bash export Beacon__EncryptionKey="" export Beacon__PreviousEncryptionKeys__0="" ``` `IStoredValueReEncryptionService` re-encrypts every stored value only a previous key decrypts (conditional writes, so a concurrent save wins). It also rewrites, in the current AES-GCM format, every value still in the legacy format written before it (unauthenticated AES-CBC), under whichever configured key. The sample host enqueues it as `ReEncryptStoredValuesJob` at every start; a host with its own jobs does the same. Startup also logs how many values remain under a previous key, how many no key decrypts, how many recipient approval hashes cannot be recomputed, and how many values are still in the legacy format. Remove the previous keys once none remain. ### The legacy format Values in the legacy format stay readable while `Beacon:Encryption:AllowLegacyCbc` is `true` (the default); the first one read is logged once as a Warning. Once the re-encryption run reports none left in the legacy format (`StoredValueReEncryptionResult.LegacyFormatReadRemovable`), set it to `false`: a legacy value is then refused like one no key decrypts. The report covers Beacon's own stored values only (data sources, repository tokens, sensitive settings, recipients); values your host stored itself through `IEncryptionService` are not scanned, so check those before turning it off. Before changing the key, set `Beacon:Mcp:CallerHashKey` to the key in use now, so MCP audit rows keep grouping together across the change. The MCP audit's content hashes (`ContentHash`, `SqlHash`) come from the same secret and behave the same way. **Set `Beacon:Audit:IntegrityKey` before you change the key** too, so security audit events written from then on do not depend on it; events signed before keep verifying while their secret stays configured (see [Security audit trail](#security-audit-trail)). ## Data Protection (the keys behind the session cookie) ASP.NET Core Data Protection signs and encrypts the `Beacon.Auth` cookie and the antiforgery tokens. It belongs to the **host**: `AddBeaconCookieAuthentication` configures none of it. | Key | Type | Default | What it does | |---|---|---|---| | `Beacon:DataProtection:ApplicationName` | string | `Beacon` in Development; **required** elsewhere | Used by `AddBeaconDataProtection(configuration, environment)`, which keeps the key ring in the Beacon database (`data_protection_keys`). Give each deployment its own name (`beacon-production`, `beacon-uat`) so a cookie from one is refused by another. Missing outside Development: startup fails | | `Beacon:DataProtection:AllowUnencryptedKeys` | bool | `false` | Acknowledges a key ring persisted without encryption outside Development (a startup Warning instead of a refusal) | - **Keys encrypted at rest.** Outside Development the host does not start when the key ring is persisted without key encryption. Chain `ProtectKeysWithAzureKeyVault(...)` or `ProtectKeysWithCertificate(...)` with a key only that environment holds, or acknowledge another protection with `AllowUnencryptedKeys`. - **Keys written before encryption are revoked.** At each start outside Development with key encryption on, keys still stored unencrypted are revoked and replaced (`DataProtectionKeyRing`, 7121); their users sign in again once. - **No repository configured** gets a startup Warning outside Development. - **Your own setup.** Call `AddDataProtection()` with your own application name, `PersistKeysTo*` and `ProtectKeysWith*`, and skip `AddBeaconDataProtection`. ```csharp builder.Services.AddBeaconDataProtection(builder.Configuration, builder.Environment) .ProtectKeysWithAzureKeyVault(new Uri(builder.Configuration["DataProtection:KeyUri"]!), new DefaultAzureCredential()); ``` **After restoring a Beacon database into another environment:** give every user a new security stamp (sessions signed in before end), delete the copied key rows so the copy creates its own keys, and if a copy may have been exposed, revoke the source environment's keys too. ```sql -- PostgreSQL UPDATE beacon.users SET security_stamp = md5(random()::text || clock_timestamp()::text || id::text); -- SQL Server UPDATE beacon.Users SET SecurityStamp = LOWER(REPLACE(CONVERT(nvarchar(36), NEWID()), '-', '')); ``` ## Authentication Beacon's sessions use the `Beacon.Auth` cookie (`HttpOnly`, `SameSite=Lax`, `SecurePolicy=SameAsRequest`, sliding; `Beacon:Authentication:Cookie:SecureAlways` set to `true` always issues it with the `Secure` flag; this covers the `Beacon.Auth` cookie only, not the antiforgery or OIDC cookies). There is no default account: [first-run setup](https://moberghr.github.io/beacon/security/user-management/#first-run-setup) creates the super admin. Register cookie authentication with the UI's base path: ```csharp builder.Services.AddBeaconCookieAuthentication("/beacon"); // login page /beacon/login ``` ### Where Beacon credentials apply Beacon's API keys, bearer tokens and `Beacon.Auth` cookie authenticate Beacon's own routes, everything under `/beacon`, and the path prefixes listed in `Beacon:Authentication:AdditionalPaths`. On every other route of the host, Beacon's API-key, bearer and cookie authentication do nothing, and the login and first-run redirects never send a request to Beacon's pages: the host's own authentication decides. | Key | Type | Default | What it does | |---|---|---|---| | `AdditionalPaths` | string[] | empty | More path prefixes where Beacon credentials apply, for example `/warp` for a Warp dashboard that admits Beacon admins. Matched by whole segments (`/warp` matches `/warp/jobs`, not `/warpx`), ignoring case, against the request path after the host's `PathBase`. Each entry starts with `/` and is not `/` alone, with no query or fragment; a trailing `/` is ignored. An invalid entry fails startup | ```json { "Beacon": { "Authentication": { "AdditionalPaths": [ "/warp" ] } } } ``` `app.UseBeaconAuthentication()` wires Beacon's authentication in the order it needs: the host's `UseAuthentication()`, then one Beacon step: a presented API key or bearer JWT, which wins over any session, else the `Beacon.Auth` cookie, followed by the origin check, with the security headers before them all unless the host added them earlier (`Beacon:Security`, below). Call it in place of `UseAuthentication()`, before `UseAuthorization()`. The cookie scheme (`Cookies`) applies the same scope itself. When it is the host's default authentication scheme (set as the default, or the only scheme registered), or a host policy names it, it authenticates nothing outside `/beacon` and `AdditionalPaths`: the session is refused there without a check against the user store, without renewal, and its cookie is left as it is. A host route that requires an authenticated user therefore needs the host's own scheme. :::caution[Upgrading] Beacon credentials no longer authenticate the host's own routes. A host route that relied on a Beacon session, API key or bearer token now sees an anonymous request: add its prefix to `Beacon:Authentication:AdditionalPaths`. A host that serves the Warp dashboard with Beacon's admin check (as the sample does) must list `/warp`, or the dashboard refuses every request. `/openapi/v1.json` and the `/.well-known` discovery documents are host routes and are no longer redirected to Beacon's login or setup page. Hosts that wired the four middlewares one by one keep working unchanged; `UseBeaconAuthentication()` is the shorter form. ::: ### Security headers and the origin check (`Beacon:Security`) | Key | Type | Default | What it does | |---|---|---|---| | `Csp` | `Enforce` \| `ReportOnly` \| `Off` | `Enforce` | The Content-Security-Policy on responses under `/beacon`. The other security headers are sent on all of Beacon's routes in every mode | | `OriginGuard` | `LogOnly` \| `Enforce` | `LogOnly` | Whether a state-changing, realtime-hub or anonymous Beacon request from a foreign origin is only logged or answered 403 | | `AllowedOrigins` | string[] | empty | The origins browsers reach Beacon from (`https://beacon.example.com`, with an optional port, no path). With `Beacon:PublicBaseUrl` they replace the default, the request's own origin. An invalid entry fails startup | `AddBeaconApiServices()` binds the section and `UseBeaconAuthentication()` applies it. See [Security headers and the origin check](https://moberghr.github.io/beacon/integrate/host-setup/#security-headers-and-the-origin-check). ### Session lifetime Bound from `Beacon:Authentication:Session`, validated at startup. | Key | Type | Default | Range | What it does | |---|---|---|---|---| | `RevalidateMinutes` | int | `5` | 0–1440 (`0` = every request) | How often, at most, a session is checked against the user store (user exists, enabled, not archived, same security stamp; roles replaced by the stored ones). Also how long a realtime connection lives before it is checked again (at least 1 minute) | | `AbsoluteLifetimeHours` | int | `12` | 1–8760 | A session ends this long after sign-in, however often it is renewed; caps the cookie's expiry and "remember me" | | `PersistSsoCookie` | bool | `false` | | A single sign-on session's cookie outlives the browser session. `false`: a session cookie, dropped when the browser closes; `true`: kept until it expires (within `AbsoluteLifetimeHours`) | ### Local password sign-in Bound from `Beacon:UserManagement:LocalLogin`, validated at startup. | Key | Type | Default | What it does | |---|---|---|---| | `Enabled` | bool | `true` | Password sign-in through the login form. `false`: `POST /beacon/api/auth/login` is not mapped (unless a break-glass account is listed), password providers refuse every account, and the login page shows no password form. Startup fails when nobody could sign in (no OIDC, no `BeaconApiOptions.ExternalLogin` (a host-owned login page), no break-glass account) | | `BreakGlassUserNames` | string[] | empty | Up to 3 stored user names (exact, case-sensitive) that may still sign in with a password while `Enabled` is `false`, at `/beacon/login?local=1`. A blank entry or more than 3 fail startup; a name no enabled internal user has gets a startup Warning | See [User management → Local password sign-in](https://moberghr.github.io/beacon/security/user-management/#local-password-sign-in). ### Optional: OIDC / SSO Bound by `builder.Services.AddBeaconOidcAuthentication(builder.Configuration)` from `Beacon:Authentication:Oidc`; validated at startup when `Enabled`. ```json { "Beacon": { "Authentication": { "Oidc": { "Enabled": true, "Authority": "https://login.microsoftonline.com/{tenant-id}/v2.0", "ClientId": "{client-id}", "AllowedTenants": [ "{tenant-id}" ], "RequiredRoles": [ "Beacon.User" ], "DisplayName": "Microsoft" } } } } ``` | Key | Type | Default | What it does | |---|---|---|---| | `Enabled` | bool | `false` | Turns SSO on | | `Authority`, `ClientId`, `ClientSecret` | string | Required | All three required when enabled. Authorization code flow with PKCE. Supply `ClientSecret` as `Beacon__Authentication__Oidc__ClientSecret` or from a secrets provider | | `CallbackPath` | string | `/signin-oidc` | The redirect callback | | `Scopes` | string[] | `openid`, `profile`, `email` | Requested scopes (replaces the list) | | `AllowedTenants` | string[] | empty | Entra tenant ids (`tid`) that may sign in. Startup fails when empty, unless `AllowAnyTenant` | | `AllowAnyTenant` | bool | `false` | Skip the tenant check (an identity provider without `tid`). A Warning at startup with an Entra authority | | `BlockGuests` | bool | `true` | Refuse Entra B2B guests (`acct=1`, or an `idp` naming another provider) | | `RequiredRoles` / `RequiredGroups` | string[] | empty | When either is set, a user must hold one listed app role (`roles`) or group (`groups`). Prefer app roles: Entra drops `groups` when a user is in too many | | `DefaultRoleName` | string | unset (no role) | Role for a user provisioned at first sign-in. May not be Admin or a functional role (startup fails) | | `DisplayName` | string | `SSO` | The SSO button's label | | `EnableFrontChannelLogout` | bool | `false` | Serve `/signout-oidc`, an anonymous `GET` that ends the session | | `RequireMfa` | bool | `false` | Refuse an interactive SSO sign-in whose ID token shows no MFA (`amr` contains `mfa`, or `acr` in `AcceptedAcrValues`). Covers the interactive SSO sign-in only | | `AcceptedAcrValues` | string[] | empty | `acr` values that count as MFA | | `McpJwksEndpoint` | string (URL) | None | Sample host: with SSO enabled, turns on bearer JWT with issuer = `Authority`, audience = `ClientId` and this JWKS URL. Not read by the Beacon libraries | Admission runs before anything is provisioned: tenant, then guest, then required role or group. See [User management → Single sign-on admission](https://moberghr.github.io/beacon/security/user-management/#single-sign-on-admission). ### JWT bearer Bearer tokens on the REST API and the external login API are configured in code; there is no configuration section: ```csharp builder.Services.AddBeaconJwtAuthentication(jwt => { jwt.EnableBearerAuthentication = true; jwt.Validation.JwksEndpoint = "https://login.microsoftonline.com/{tenant}/discovery/v2.0/keys"; jwt.Validation.ValidIssuer = "https://login.microsoftonline.com/{tenant}/v2.0"; jwt.Validation.ValidAudience = "{beacon-client-id}"; }); ``` | Option | Default | What it does | |---|---|---| | `ExternalLoginEndpoint` | None | The login API the login form posts to, an `https` URL; internal users sign in first (`HybridAuthenticationProvider`) | | `EnableBearerAuthentication` | `false` | Accept `Authorization: Bearer ` on the REST API | | `AccessTokenClaim` | None | For a non-Entra issuer without `typ: at+jwt`: a claim only its access tokens carry. A claim ID tokens carry stops startup | | `Validation.SigningKey` / `JwksEndpoint` | None | One is required. A signing key is at least 32 bytes (HS256/384/512 only); the JWKS endpoint is an `https` URL (RS, PS and ES 256/384/512 only). `http` to a loopback host is accepted in Development only | | `Validation.ValidIssuer(s)`, `ValidAudience(s)` | None | At least one of each is required | | `Validation.ClockSkew` | 5 minutes | At most 5 minutes | | `Validation.ValidateLifetime`, `ValidateIssuer`, `ValidateAudience` | `true` | Must stay `true` | | `ClaimsMapping.*` | `sub`, `preferred_username`, `email`, `roles`, `name` | Which claims hold the user id, user name, e-mail, roles and display name | With bearer or external login on, a missing or short key, a missing issuer or audience, a validation flag turned off, a skew above 5 minutes or an `http` endpoint stops the host. A token must be an access token and name an existing, enabled, external Beacon user; see [User management](https://moberghr.github.io/beacon/security/user-management/#jwtexternalapiauthenticationprovider). Entra callers on `/beacon/mcp` are configured separately (below). ### MCP callers from Entra ID (`Beacon:Mcp:Callers`) Validated at startup. Without the section, no JWT caller is admitted on `/beacon/mcp`. Active callers also need bearer JWT enabled with issuer and audience validation. | Key | Type | Default | What it does | |---|---|---|---| | `AllowedTenants` | string[] | empty | Tenants whose tokens map to a caller; required when callers are active | | `Users:Enabled` | bool | `false` | Accept delegated user tokens | | `Users:RequiredRoles` / `RequiredGroups` | string[] | empty | Any-of admission rules; both empty fails startup unless `AllowAnyTenantUser` | | `Users:AllowAnyTenantUser` | bool | `false` | Admit every user of the allowed tenants | | `Users:ProjectIds` | int[] | empty | Projects for every accepted user | | `Users:GroupProjects` | map group or role → int[] | empty | Extra projects per group or app role | | `Users:Scope` | `Read` \| `Execute` | `Execute` | Scope of user callers | | `Users:AutoProvision` | bool | `true` | Find or create a Beacon user keyed on `oid` | | `Users:DefaultRoleName` | string | `Viewer` | Role for provisioned users; not Admin or a functional role | | `Systems[i]:Name` | string | Required | Required, unique | | `Systems[i]:ClientId` or `ObjectId` | string | Required | Exactly one: an app-only token's client, or a service user's `oid` | | `Systems[i]:ProjectIds`, `Scope`, `HostClaims[]` | | `Execute` | The system caller's projects, scope and host-endpoint claims | See [Entra ID callers](https://moberghr.github.io/beacon/mcp/entra-callers/). ## Data-source transport security Every data-source connection must authenticate the server over TLS (for example `SSL Mode=VerifyFull` on PostgreSQL, `SslMode=VerifyFull` on MySQL, `Encrypt=Mandatory` or `Strict` without `TrustServerCertificate=true` on SQL Server and Azure Synapse). The rule per engine is in [Transport security (TLS)](https://moberghr.github.io/beacon/features/data-sources/#transport-security-tls). Creating a data source, changing its connection and testing a connection refuse a connection that does not meet it. Bound from `Beacon:DataSources:Transport`: | Key | Type | Default | What it does | |---|---|---|---| | `EnforceOnExisting` | bool | `false` | Refuses a stored data source whose connection does not meet the rule every time it connects. While `false`, such a source keeps working, the data-source list reports it (`transportVerified: false`) and the log carries one Warning per data source per process | | `AllowInsecure` | bool | `false` | Lifts the rule for new, changed and stored connections alike. For development hosts whose local databases have no TLS; outside Development the host logs a Warning at startup while it is set. The sample host sets it in `appsettings.Development.json` only | ```json { "Beacon": { "DataSources": { "Transport": { "EnforceOnExisting": true } } } } ``` Host-managed data sources follow the same rules, applied to the `ConnectionStrings:{Name}` value each time they connect. That value is checked as a stored connection, never refused as a new one: a value that does not meet the rule is logged as a Warning, and refused only while `EnforceOnExisting` is `true`. :::caution[Upgrading] PostgreSQL and MySQL data sources stored without `VerifyFull`, and SQL Server or Azure Synapse sources with `Encrypt=False` or `TrustServerCertificate=True`, keep working but are flagged and logged once per process; new ones are refused. Replace each flagged connection, then set `EnforceOnExisting` to `true`. A development host that connects to local databases without TLS sets `AllowInsecure` in its Development configuration only. ::: ## Data-source driver options and hosts Each connector also refuses driver options that act on the Beacon host or sign in as it (the list per engine is in [Driver options and hosts](https://moberghr.github.io/beacon/features/data-sources/#driver-options-and-hosts)), on create, change and test connection; a stored source gets one Warning per process instead, and is refused under `Beacon:DataSources:Transport:EnforceOnExisting`; a host-managed source gets the host list only. `Beacon:DataSources:Transport:AllowInsecure` lifts the option rule, never the host list. Bound from `Beacon:DataSources`: | Key | Type | Default | What it does | |---|---|---|---| | `AllowedHosts` | string[] | `[]` | The hosts a data-source connection may reach: exact host names or IP addresses, or `*.example.com` for the hosts below `example.com` (ASCII; an internationalised name in its `xn--` form). Empty leaves hosts unrestricted. While set, a connection reaching an unlisted host, a host that is not a bare host name or IP address, or one whose hosts cannot be told, is refused; a BigQuery source needs `oauth2.googleapis.com`. Resolved addresses are not pinned. An entry with a scheme, port or path fails startup | | `AllowHostCredentials` | bool | `false` | Accepts connections that sign in as the Beacon host: integrated or Kerberos security, a managed, workload or default cloud identity, an AWS profile or IAM role, BigQuery application default credentials, a certificate store, a local socket or named pipe, a PostgreSQL pgpass file | ```json { "Beacon": { "DataSources": { "AllowedHosts": [ "*.db.example.com", "adb-1234567890123456.7.azuredatabricks.net" ] } } } ``` :::caution[Upgrading] New data sources that sign in as the Beacon host (for example SQL Server `Integrated Security=True`, CloudWatch with the host's IAM role, BigQuery without service account JSON) are refused unless `AllowHostCredentials` is `true`; stored ones keep working with a Warning. BigQuery sources whose service account JSON is not a service account key, or whose key names a token endpoint, universe or URL that is not Google's, stop connecting. New MySQL sources without a password are refused. PostgreSQL connections with a password no longer answer a server's Kerberos or SSPI request: set `Require Auth` yourself (and `AllowHostCredentials` for an Admin's source) if a password-carrying connection must sign in with Kerberos. ::: ## Authorization Bound from `Beacon:Authorization`, validated at startup. Whether authorization is on at all is code-only (`options.Authorization.Enabled`). See [Authorization](https://moberghr.github.io/beacon/security/authorization/). | Key | Type | Default | What it does | |---|---|---|---| | `RequireProjectMembership` | bool | `false` | Restrict every project to its members and Admins; data sources no project links become Admin-only. Startup fails without a membership source (Beacon user management or a host `IProjectMembershipSource`) | ### Functional permissions: role names, identity-provider app roles, separation of duties ```json { "Beacon": { "Authorization": { "Roles": { "Auditor": "Compliance Reviewer", "HonourHostRoleClaims": false }, "IdpRoles": { "Enabled": true, "Map": { "beacon.auditor": "Auditor" }, "ClaimLifetimeMinutes": 60 }, "SeparationOfDuties": { "Enabled": true } } } } ``` | Key | Type | Default | What it does | |---|---|---|---| | `Roles:Approver`, `Auditor`, `SecurityAdmin`, `UserAdmin`, `DataSourceAdmin`, `MigrationOperator` | string | the permission's name | The level-0 system role that grants the permission. At most 50 characters, not `Admin`, `Editor` or `Viewer`, unique across permissions (ignoring case); blank keeps the default | | `Roles:HonourHostRoleClaims` | bool | `false` | Host **without** Beacon user management: let session role claims named like a permission's role grant it. No effect with user management | | `IdpRoles:Enabled` | bool | `false` | Map identity-provider app roles to permissions at SSO sign-in (session claims only, never stored). Needs at least one `Map` entry | | `IdpRoles:Map` | map app role → permission | empty | Only the six permissions; never Admin, Editor or Viewer | | `IdpRoles:ClaimLifetimeMinutes` | int | `60` | 1–1440. How long a mapped permission counts after the sign-in | | `SeparationOfDuties:Enabled` | bool | `false` | Refuse (`400`) assignments giving one user Approver + SecurityAdmin, Auditor + SecurityAdmin, Auditor + UserAdmin, UserAdmin + Approver or UserAdmin + MigrationOperator; an `IdpRoles:Map` onto such a pair fails startup | ## User management and first-run setup User management itself is code-only ([Code-only options](#code-only-options)); these keys are bound from configuration: | Key | Type | Default | What it does | |---|---|---|---| | `Beacon:UserManagement:SetupToken` | string | unset | The secret first-run setup asks for. At least 32 characters after trimming, or startup fails (checked when user management is on). Unset: Beacon generates a token per process and prints it once to the process console (standard error, never the log) while no super admin exists. Set it when several replicas serve the setup page | | `Beacon:UserManagement:LocalLogin:*` | | | See [Local password sign-in](#local-password-sign-in) | Login attempts are throttled at 10 per 60 seconds per client address; the first-run setup endpoints have their own budget. Neither is configurable. ## API keys Bound from `Beacon:ApiKeys`, validated at startup. See [API keys](https://moberghr.github.io/beacon/security/api-keys/). | Key | Type | Default | Range | What it does | |---|---|---|---|---| | `MaxLifetimeDays` | int | `365` | 1–3650 | Longest lifetime a new key may have; a key created without a date gets 90 days, capped at this | | `EnforceMaxLifetimeOnExistingKeys` | bool | `false` | | A key stored without an expiry expires `MaxLifetimeDays` after creation. Off: such keys keep working and warn once per process | | `MaxOwnerInactivityDays` | int | `0` (off) | 0–3650 | A key stops working (`401`) while its owner has not signed in with their own identity for this many days | Scopes are `Read` and `Execute`; a key stored with the retired `Admin` scope is read as `Execute`. ## Query execution row cap Bound from `Beacon:Query`, validated at startup. These apply to every statement Beacon runs for its own features: previews, subscription runs, AI actor checks, ad-hoc queries. | Key | Type | Default | Range | What it does | |---|---|---|---|---| | `MaxExecutionRows` | int | `100000` | 1–10,000,000 | The most rows one statement may return. The cap is written into the SQL (a larger `LIMIT`/`TOP`/`FETCH` is lowered to it); on PostgreSQL, MySQL, SQL Server, Azure Synapse and Snowflake the read also stops one row past it, BigQuery stops reading the job's result there, and Databricks returns no more than that (`row_limit`). A larger result **fails** the run, never truncates, with a message naming the key. Rows are held in memory | | `StatementTimeoutSeconds` | int | `120` | 1–3600 | Cancels one statement: a query step, a final query on the in-memory join, a metadata sample. A subscription's own timeout may only be shorter (saving a longer one is refused). On Snowflake the session also runs under `STATEMENT_TIMEOUT_IN_SECONDS` of this value, so the server stops the statement too | | `MaxResultBytes` | long | `268435456` (256 MiB) | greater than 0 | The most bytes one statement's result may hold while it is read from a PostgreSQL, MySQL, SQL Server, Azure Synapse or Snowflake data source, capped or not, and while a final query's result is read from the in-memory join. Estimated as the rows arrive: each row 32 bytes plus 8 a column, each value at least 8 bytes (text two bytes a character, binary its length, a number, date, time or identifier 16, an array, list, record or map its elements however deeply nested, any other value two bytes a character of its text form). A larger result **fails** the run as soon as it passes the limit, with "The result is larger than the N-byte limit (Beacon:Query:MaxResultBytes); select fewer columns or rows."; no value or row is cut short. One row (on PostgreSQL, one value) is read whole before it is counted, and the rows are copied on their way to the caller, so peak memory can be a few times the limit. Not the MCP **Max result size** (`maxResultBytes` in the MCP settings), which bounds the result an MCP tool returns to its client — see [MCP server](https://moberghr.github.io/beacon/mcp/server/#configuration) | Opening a connection to a PostgreSQL, MySQL, SQL Server or Azure Synapse data source waits at most 30 seconds: a connection string that sets no connect timeout (`Timeout`, `Connection Timeout`, `Connect Timeout`), or `0`, gets 15 seconds, and a longer one is lowered to 30. Snowflake keeps its driver's own login timeout. Data migration reads use `Beacon:Migrations:MaxRows` instead. The MCP tools that run SQL use the project's MCP statement timeout (30 seconds by default). | Key | Type | Default | Range | What it does | |---|---|---|---|---| | `Beacon:Metadata:ForcedRefreshCooldownMinutes` | int | `15` | 0–1440 (`0` = off) | After a successful forced metadata refresh of a data source, the next one answers `429` until this passes. Per process; Admins and DataSourceAdmins are not held to it. Validated at startup | | `Beacon:AiActors:ThinkCooldownMinutes` | int | `15` | 0–1440 (`0` = off) | An AI actor's think cycle starts at most once per this many minutes, from its last think time. Background cycles within it are skipped; an on-demand cycle is refused with the time left. Validated at startup | | `Beacon:AiActors:PlanExpiryDays` | int | `7` | 1–365 | An AI actor's plan awaiting approval expires this many days after it was proposed: it is no longer listed as pending, and approving, rejecting or revising it answers `409`. Computed when read. Validated at startup | ## Request limits Bound from `Beacon:RateLimits`, validated at startup. Each limit is a fixed window: `PermitLimit` requests per `WindowSeconds` (1–3600, default `60`) in each partition; `PermitLimit` is 0–100,000 and `0` turns that limit off. An authenticated caller is one partition per credential (an API key by its id, a mapped MCP caller by its subject hash, a signed-in user), an anonymous one per client address: an IPv4 address (also one mapped into IPv6) whole, an IPv6 address by its `/64` block, so one client cannot spread its requests over the addresses of its block. Each limit keeps at most `Beacon:RateLimits:MaxPartitions` partitions (default `10000`, 100–1,000,000); a new one beyond that replaces the one used least recently, which starts over with a full budget. The address is `HttpContext.Connection.RemoteIpAddress`, which `UseForwardedHeaders` rewrites only as your host configures it: behind a reverse proxy without forwarded headers every caller shares the proxy's partition (see [Production checklist](https://moberghr.github.io/beacon/integrate/production/#reverse-proxy)). Beacon's endpoints apply the limits themselves, on Beacon's routes only; `AddRateLimiter`/`UseRateLimiter` are not needed and a host's own limiter does not change them. The counts are kept in memory, so with N instances behind a load balancer a caller can reach N times a limit. Over a limit the answer is `429` as `application/problem+json` with `Retry-After` (whole seconds). | Key | Default | Partition | What it limits | |---|---|---|---| | `Login` | `10` a minute | client address | Password sign-in (`POST /beacon/api/auth/login`), and first-run setup in its own partition | | `Api` | `600` a minute | caller (anonymous: address) | Every `/beacon/api` request | | `Anonymous` | `30` a minute | client address | Anonymous requests to Beacon's routes answered `401` (no credential, an unknown or expired API key or bearer token); over it they get `429` instead | | `Mcp` | `60` a minute | API key or MCP caller | Every request to `/beacon/mcp` | | `Ask` | `10` a minute | caller | The `ask` tool, through MCP and the playground; over it the tool answers an error (audited like every refusal) | | Key | Type | Default | Range | What it does | |---|---|---|---|---| | `Beacon:RateLimits:Lockout:Enabled` | bool | `true` | | Apply the lockout. `false` (or `MaxFailedAttempts` `0`) turns it off: password sign-in is then limited only per client address by `Login` | | `Beacon:RateLimits:Lockout:MaxFailedAttempts` | int | `5` | 0–100 (`0` = off) | After this many wrong passwords an internal user's password sign-in, and the password change of a signed-in user, is locked. Counted per user (stored on the user, so it holds across instances), a wrong current password on a password change included; a name no user has is counted against nobody. The break-glass accounts (`Beacon:UserManagement:LocalLogin:BreakGlassUserNames`) are never counted or locked: they stay limited per client address by `Login` | | `Beacon:RateLimits:Lockout:DurationMinutes` | int | `15` | 1–1440 | How long the lock lasts. While locked every password, the right one included, gets the same generic answer as a wrong one, after the same password check; wrong passwords while locked do not extend it. The lock is decided after the password check, together with the write that clears the count, so attempts already in flight when another attempt locks the user are refused too. A successful sign-in, the lock itself, an administrator's password reset and a wrong password sent this long or longer after the previous one start the count again. A lock records `auth.user_locked_out` and logs `UserLockedOut` (7124); a refused attempt records `auth.password_sign_in_refused` with reason `locked_out` | The lockout counts per user, not per client: anyone who knows a user name can lock that user's password sign-in by sending wrong passwords, and again each time the lock ends (the per-address `Login` limit slows this down, but several addresses get past it). That denial of service is the price of a hard cap on password guessing. Keep a break-glass account for emergencies (it is never locked), prefer single sign-on for everyday users, or set `Lockout:Enabled` to `false` when the denial of service weighs more for your deployment than the cap. ## Request size and time limits Bound from `Beacon:RequestLimits`, validated at startup. They apply to Beacon's own routes only (`/beacon/api`, the hub under it and `/beacon/mcp`), through `UseBeaconAuthentication()`, and a host's own limit wins where it is stricter: a lower Kestrel `MaxRequestBodySize` is never raised. `0` turns one bound off. They do not touch the 5-minute Kestrel keep-alive and header timeouts or the 5-minute `HttpClient` timeout that AI calls need (see [Host setup](https://moberghr.github.io/beacon/integrate/host-setup/)). | Key | Type | Default | What it does | |---|---|---|---| | `ApiMaxBodyBytes` | long | `2097152` | Largest request body on `/beacon/api`; a larger one answers `413` | | `McpMaxBodyBytes` | long | `4194304` | Largest request body on `/beacon/mcp` | | `ApiTimeoutSeconds` | int | `300` | A `/beacon/api` request still running after this has its cancellation token cancelled and, if nothing was sent yet, answers `504`. The hub and the routes that run long work inline (migration run, metadata refresh, documentation generation, repository scan, eval run, playground tool run) are exempt and keep their own timeouts. A host's own long route opts out with `.WithBeaconLongRunning()` | | `HubMaxReceiveMessageBytes` | long | `8192` | Largest message the hub takes from a client (it has no client-callable methods) | | `HubMaxParallelInvocationsPerClient` | int | `1` | Hub invocations in flight per connection | | `HubClientTimeoutSeconds` | int | `30` | Silence after which the hub drops a connection | | `McpMaxIdleSessions` | int | `1000` | Idle MCP sessions kept in memory (`HttpServerTransportOptions.MaxIdleSessionCount`; the SDK default is 10,000). A session is dropped after 30 idle minutes | | `McpMaxInitializesPerCaller` | int | `60` | MCP `initialize` requests one caller (API key, mapped caller, user or address) may send per window; more answer `429` with `Retry-After`. Per instance, in memory | | `McpInitializeWindowSeconds` | int | `60` | The window of the previous key | A host that configures SignalR (`BeaconApiOptions.ConfigureSignalR`, `AddHubOptions`) or `HttpServerTransportOptions` itself overrides the hub and idle-session values. ## Subscription limits Bound from `Beacon:Subscriptions`, validated at startup. | Key | Type | Default | Range | What it does | |---|---|---|---|---| | `MinIntervalMinutes` | int | `5` | 1–1440 | The shortest time between two runs of a subscription's cron schedule (in UTC). Creating a subscription with a schedule that fires more often (an AI actor's included), or changing one to it, is refused; a subscription saved before keeps its schedule through other edits and a reactivation | | `MaxAttachmentBytes` | int | `10485760` (10 MB) | 65,536–104,857,600 | The largest file a subscription attaches to its email. A larger CSV or XLSX keeps only its first rows that fit (a CSV exactly as many; an XLSX file can keep somewhat fewer), and the email says how many of the rows it delivers the file holds; a first row larger than the limit leaves the column headers only, and headers larger than the limit leave no attachment. The email itself is about 1.33 times the attachment's size (base64 encoding), so keep the limit below your mail server's message limit | See [Subscriptions](https://moberghr.github.io/beacon/features/subscriptions/). ### Reports (`Beacon:Reports`) | Key | Type | Default | What it does | |---|---|---|---| | `NeutralizeFormulas` | bool | `true` | A text cell of a CSV or XLSX attachment that starts with `=`, `+`, `-`, `@`, a tab or a carriage return is written so a spreadsheet reads it as text: a CSV cell gets a leading `'`, an XLSX cell the quote prefix. Numbers, dates and booleans are never touched. Set `false` only when the file is not opened in a spreadsheet | ## Data migration Bound from `Beacon:Migrations`, validated at startup. **A deployment lock**: only configuration turns it on. | Key | Type | Default | What it does | |---|---|---|---| | `Enabled` | bool | `false` | Maps the migration endpoints, shows the feature, lets jobs run and lets an Admin mark data sources as migration write targets. Off: nothing runs, for Admins too | | `MaxRows` | int | `100000` | 1–10,000,000. One run's read and write budget; a larger source fails before anything is written | See [Data migration](https://moberghr.github.io/beacon/features/data-migration/). ## Approval workflow The four-eyes workflow is **off** unless the host turns it on, in code. See [Approvals](https://moberghr.github.io/beacon/security/approvals/) for what it governs. ```csharp builder.Services.AddBeaconServices(builder.Configuration, options => { // ... options.EnableApprovalWorkflow(x => { x.RequireApprovedVersionToRun = true; // strict mode }); }); ``` | Option (`options.ApprovalWorkflow.*`) | Default | What it does | |---|---|---| | `Enabled` | `false` | Every change to a query's executable content waits for a second person's approval; subscriptions run only reviewed SQL | | `AllowSelfApproval` | `false` | Lets the requester approve their own change (for a host with one approver) | | `RequireApprovedVersionToRun` | `false` | Strict mode: a subscription or saved-query MCP tool runs only a version approved for exactly its content | | `CoversMigrationJobs` | `true` | While the workflow is on, a new migration job runs only after approval | | `CoversCustomSqlRules` | `true` | While the workflow is on, a new or changed Custom SQL rule is evaluated only after approval | | `CoversRecipientDestinations` | `true` | While the workflow is on, a new recipient or changed destination receives nothing until approved | | `CoversDataSourceConnections` | `false` | While the workflow is on, a new data source or a connection changed in anything but its secrets connects to nothing until approved | | `CoversLlmConnection` | `false` | While the workflow is on, a change of the stored LLM connection's provider, endpoint, region or authentication mode waits for a live Admin's approval; the stored connection stays in use meanwhile | | `CoversAdminRoleGrants` | `false` | While the workflow is on, giving a user the Admin role (a new user or a role change) waits until another live Admin approves it, whatever `AllowSelfApproval` says; removing it, and functional roles, apply at once | | `LlmConnectionRequestDays` | `7` | How long a staged LLM connection change waits for a decision before it expires and what it staged is wiped; held between 1 and 30 | The `Covers…` switches apply only while `Enabled` is on. Turning a switch, or the workflow, off releases what it held; startup warns with the counts. Startup also logs how changes are governed (a Warning outside Development while the workflow is off). ## Security audit trail Beacon records sign-ins and changes to identities, keys, access, data sources, delivery, settings, approvals, ownership, tasks, curation, data quality and migrations in an append-only, signed trail an Auditor reads (see [Security audit trail](https://moberghr.github.io/beacon/security/audit-trail/)). It needs no configuration; these settings tune it: ```json { "Beacon": { "Audit": { "IntegrityKey": "", "PreviousIntegrityKeys": [], "RetentionDays": null } } } ``` | Key | Type | Default | What it does | |---|---|---|---| | `Beacon:Audit:IntegrityKey` | string | derived | The secret events are signed with: at least 32 characters, supplied like the encryption key (a secret store, or `Beacon__Audit__IntegrityKey`), never committed. Unset, Beacon derives a key for this purpose from `Beacon:Mcp:CallerHashKey`, else from `Beacon:EncryptionKey`. Set it before you change the encryption key | | `Beacon:Audit:PreviousIntegrityKeys` | string[] | empty | Integrity keys used earlier, as a list (`Beacon__Audit__PreviousIntegrityKeys__0`, ...): accepted when verifying events, never used to sign. Move the old key here when you set a new one | | `Beacon:Audit:RetentionDays` | int | none (keep forever) | Days events are kept: between `365` and `36500`. The purge must be scheduled; the sample host runs it daily as the Warp job `security-audit-retention` (see [Retention](https://moberghr.github.io/beacon/security/audit-trail/#retention)) | An integrity key shorter than 32 characters or containing `${`, or a retention outside the range, stops the host at startup with a message that never contains a key. At startup the host logs the id of the key events are sealed under and where it comes from (never the key), with a Warning while `IntegrityKey` is unset. Two jobs keep the trail checked and bounded; neither runs unless the host schedules it. The sample host schedules both as Warp recurring jobs: `security-audit-verify` (hourly at `:15`), which verifies the events written since its last run and writes a checkpoint to the `Beacon.Audit` log stream, and `security-audit-retention` (daily), the purge. A host embedding Beacon schedules `ISecurityAuditVerificationService.VerifyNewEventsAsync` and `ISecurityAuditRetentionService.PurgeExpiredAsync` itself (see [Scheduled verification](https://moberghr.github.io/beacon/security/audit-trail/#scheduled-verification)). Alert on the counters `beacon.security_audit.write_failures` and `beacon.security_audit.verification_failures` above zero. ## AI / LLM Bound from `Beacon:LLM` by `builder.Services.AddBeaconAI(builder.Configuration)`. Optional; not validated. AI features also need `options.UseAI = true`. | Key | Type | Default | What it does | |---|---|---|---| | `Provider` | `OpenAI` \| `Claude` \| `AzureOpenAI` \| `Bedrock` | `OpenAI` | The provider (`Claude` is Anthropic). `Bedrock` needs `Moberg.Beacon.AI.Bedrock` + `AddBeaconBedrock()` | | `ApiKey` | string | `""` | The provider key. Bedrock: a legacy `accessKey:secretKey` pair, used when `AwsAccessKeyId` and `AwsSecretAccessKey` are both empty (under `IamRole` with a logged warning) | | `BedrockAuthMode` | `IamRole` \| `AccessKey` \| `TemporaryCredentials` | `IamRole` | Bedrock credentials: the default AWS credential chain, the access key pair, or the pair with `SessionToken` | | `AwsAccessKeyId`, `AwsSecretAccessKey` | string | None | The Bedrock access key pair for `AccessKey` and `TemporaryCredentials`; one without the other fails | | `Endpoint` | string | None | Required for `AzureOpenAI` | | `Region` | string | None | Required for `Bedrock` | | `SessionToken` | string | None | Bedrock temporary credentials | | `Model` | string | `""` | The model; for Azure the deployment name, for Bedrock the model id | | `FastModel` | string | None | Model for the short classification calls on the `ask` path (intent classification and data-source routing). Empty or equal to `Model`: those calls use `Model`, as does every other call. If the fast model fails when called, the call is served once on `Model` | | `Limits:MaxConcurrentRequests` | int | `50` | Concurrency of the LLM request queue; a value saved in Admin Settings replaces it without a restart. Background work (documentation generation, indexing, learning, scheduled actor cycles, eval judging) holds at most half the slots (at least one) | | `Limits:InteractiveQueueWaitSeconds` | int | `60` | How long an interactive call waits for a free slot before it fails with *The LLM is busy. Try again later.* (`LlmQueueSaturatedException`). Non-positive: the default; at most `3600` | | `Limits:BackgroundQueueWaitSeconds` | int | `300` | The same wait for background work. Non-positive: the default; at most `3600` | | `Limits:MaxQueuedRequests` | int | `200` | How many interactive calls may wait for a slot at once; background work may have half as many waiting (at least one). One more fails as busy at once. Non-positive: the default | | `Limits:RateLimitsEnabled` | bool | `false` | Enforce `TokensPerMinute` and `RequestsPerMinute` (a per-minute token bucket each; a call that cannot get its share within its queue wait fails as busy). Admin Settings' **Enforce requests/tokens per minute** replaces it once saved; like the other limits saved there, the stored value is applied again at startup | | `Limits:TokensPerMinute`, `RequestsPerMinute` | int, int | `80000`, `1000` | Applied only while `RateLimitsEnabled` is on; `0` or less is unlimited. A call reserves its prompt characters / 4 plus its maximum answer tokens and settles them with the reported usage | | `Limits:MonthlyBudget` | decimal | `100.00` | Recorded, not enforced (Beacon keeps no record of spend) | ```json { "Beacon": { "LLM": { "Provider": "Claude", "Model": "", "FastModel": "" } } } ``` ```bash export Beacon__LLM__ApiKey="" ``` At startup the provider is built from `Beacon:LLM`. Saving the AI provider in [Admin settings](https://moberghr.github.io/beacon/security/admin-settings/) swaps it immediately, without a restart; after a restart `Beacon:LLM` applies again until the settings are saved. Keep `Beacon:LLM` in step with Admin settings, or save them again after a restart. See [AI providers](https://moberghr.github.io/beacon/integrate/ai/). ### Local embeddings (`Beacon:Embeddings`) | Key | Type | Default | What it does | |---|---|---|---| | `Enabled` | bool | `false` | Run a local ONNX embedding model in-process for semantic retrieval | | `ModelPath` | string | None | The ONNX model file (384 dimensions, for example bge-small-en-v1.5) | | `TokenizerPath` | string | None | The WordPiece vocabulary file | These keys are read by the `Moberg.Beacon.AI.Embeddings.Onnx` package (`AddBeaconOnnxEmbeddings()`). The service is available only when `Enabled` is `true` and both files exist; otherwise retrieval falls back to lexical matching. On PostgreSQL the vectors live in a pgvector column, so the `vector` extension must be available to Beacon's migrations; on SQL Server they are compared in memory. The sample host re-indexes every 12 hours (`mcp-embedding-reindex`, `mcp-docchunk-reindex`). ## GitHub repository scan (`Beacon:GitHub`) A project's repository scan reads source files from `https://github.com` with the repository's stored token. Only `https://github.com/owner/repo` URLs (or `owner/repo`) are scanned; a URL on another Git host (GitHub Enterprise Server, GitLab, Azure DevOps) fails the scan with a message saying so, and no request is made. Every scan is bounded: | Key | Type | Default | Meaning | |---|---|---|---| | `MaxFilesPerRepository` | int | `5000` | The most source files read from one repository. The rest are left out | | `ScanTimeout` | timespan | `00:05:00` | The longest one repository scan runs. A scan that reaches it keeps what it found | | `MaxFileBytes` | int | `1000000` | A larger source file is skipped | | `MaxTreeResponseBytes` | int | `33554432` | The most bytes read from the repository's file list | | `TaskDeadline` | timespan | `03:00:00` | The longest a background scan of a project's repositories runs: no repository's scan starts after it, and the ones left count as not scanned. More than zero and at most 24 hours, checked at startup | A scan that hits a bound still ends as completed, and its error text starts with "Partial scan:" and names what was left out (files over the cap, files over the size limit, the time limit, files that could not be read, or a file list GitHub itself truncated), so a partial result is never mistaken for a full one. The project's repository list marks such a repository **Partial** with the reasons (`partialScan` and `partialScanReasons` on each repository of `GET /beacon/api/projects/{id}`), and `POST /beacon/api/projects/{id}/scan` returns them in `partialScans`. **Scan repositories** on a project's Repositories tab (an Editor's) starts the scan as a background task, `POST /beacon/api/projects/{id}/scans` (`202` with `{ taskId, jobId, state }`; progress at `GET /beacon/api/projects/{id}/scans/{taskId}`, whose `warningCount` is the repositories that failed, were only partly scanned or were not scanned before `TaskDeadline`): it runs as the user who started it, checks again that they may still change the project (`forbidden` otherwise), and each repository keeps these bounds. One scan of a project is queued or running at a time (`409` for a second). Before each repository the task records that it is still going; one silent for `TaskDeadline` plus `ScanTimeout` plus 15 minutes is given up (`abandoned`), and a task given up stops before its next repository. The scan is on record when it ends (`access.repository_scan_ran`, with the ids of the repositories it scanned then, which can include one added after the start, and its counts; `refused` with the reason when it was skipped). The in-request `POST /beacon/api/projects/{id}/scan` stays for Beacon 5.0 callers. The time limit also covers reading the repository's metadata and file list: a scan that reaches it before the file list is read fails. A scan replaces the repository's code references only when it finishes; one that fails or is cancelled by the caller keeps the previous references. ## Notification destinations (optional) Bound from `Beacon:Notifications`, validated at startup (each bad entry is named); the effective policy is logged once. | Key | Default | What it does | |---|---|---| | `AllowedHosts:Slack` / `Teams` / `Jira` | empty | Extra hosts on top of the built-in ones: Slack `hooks.slack.com`; Teams `*.webhook.office.com`, `*.logic.azure.com`, `*.environment.api.powerplatform.com`; Jira `*.atlassian.net`, `api.atlassian.com`. `*.example.com` matches subdomains only; `*.com` is refused | | `AllowedHosts:Webhook` | empty (any public host) | When set, the only hosts a webhook may target | | `AllowedHosts:Email` | empty (any domain) | When set, the only e-mail domains | | `DisabledTypes` | empty | Types this host does not deliver: `Email`, `Teams`, `Slack`, `Jira`, `Webhook` | | `AllowedPrivateNetworks:Webhook` / `Slack` / `Teams` / `Jira` | empty | IP addresses, CIDR ranges or host names that calls of that type may still reach though private | | `AllowedHeaders` | empty | Webhook header names allowed besides `Authorization`, `Api-Key`, `X-Api-Key`, `Ocp-Apim-Subscription-Key`. Reserved names (`Host`, `Content-Length`, `X-Forwarded-*`, `Metadata-Flavor`, …) are refused | | `Nat64Prefixes` | empty | Extra NAT64 `/96` prefixes besides `64:ff9b::/96` | | `UseSystemProxy` | `false` | Send notification calls through the system proxy, with the destination checked before sending. Requires `AllowedHosts:Webhook` (or Webhook disabled) | | `RequireEncryptedSecrets` | `false` | Refuse to send to a recipient whose secrets are still stored unencrypted | Slack, Teams, Jira and webhook destinations need `https` without user credentials. Calls never follow redirects, time out after 30 seconds (10 seconds to connect), and by default connect directly, only to public addresses Beacon resolved and checked itself. :::caution[Proxy mode] Through a proxy, the proxy resolves the host name and connects to the address it resolves. Configure the proxy's egress policy to refuse internal, link-local and cloud-metadata destinations, and keep `AllowedHosts:Webhook` short. ::: Recipients saved by an older version keep working with unencrypted secrets until saved again or until `IRecipientSecretEncryptionService.EncryptStoredSecretsAsync` runs (the sample's `EncryptRecipientSecretsJob`, which the host enqueues once every node runs the new version; an earlier version cannot read the encrypted secrets). Then set `RequireEncryptedSecrets` to `true`. Notification URLs are secrets: leave notification calls out of HTTP tracing (`NotificationRequests.IsNotificationRequest`). See [Notifications](https://moberghr.github.io/beacon/features/notifications/). ## Email adapter Beacon's libraries ship no email implementation. Register your own `IEmailAdapter` with `options.AddEmailAdapter()`. The sample host's `BeaconMailSender` reads these keys, at send time: | Key | Default | What it does | |---|---|---| | `Beacon:Smtp:Host` | **required** | SMTP server | | `Beacon:Smtp:Port` | `465` | Port | | `Beacon:Smtp:EnableSsl` | `true` | TLS | | `Beacon:Smtp:Username` | **required** | Login | | `Beacon:Smtp:Password` | **required** | Password; supply as a secret | | `Beacon:Smtp:FromAddress` | **required** | Sender address | A missing required key fails the send, not the startup. These keys belong to the sample's adapter; a host with its own adapter reads whatever it needs. ## Data retention (`Beacon:Retention`) One job, `IDataRetentionService.RunAsync`, keeps Beacon's run history and logs bounded. The sample host runs it daily as the Warp job `data-retention` (one run at a time, at most an hour); a host embedding Beacon schedules it itself, with a timeout longer than the run's 50-minute budget. It first clears the **content** of the content stores' rows older than `ContentDays`, keeping the rows and their structure (counts, timings, statuses, ids), then deletes the rows of every retention target older than the target's window. ```json { "Beacon": { "Retention": { "Days": 90, // recommended; 0 (the default) keeps everything "Targets": { "subscription-runs": 365, "ai-actor-executions": 30 }, "ContentDays": 30, "Locked": false } } } ``` | Key | Type | Default | Description | |---|---|---|---| | `Days` | int | `0` (keep forever) | Days the rows of every target are kept; `0` keeps them forever. `90` is recommended | | `Targets:{name}` | int | `Days` | One target's own window; `0` keeps its rows forever. An unknown name stops the host at startup | | `ContentDays` | int | `0` (off) | Days the content of the content stores is kept: SQL text, result snapshots, model output and error text become empty. `30` is recommended for regulated deployments | | `Locked` | bool | `false` | Deployment lock: `Days` (also when every target sets its own window), every target, `ContentDays` and `Beacon:Mcp:Audit:RetentionDays` must be greater than 0, or the host fails to start; the learning-signal retention Admins edit at runtime never resolves above `Days`. What it leaves out by design is listed below | | Target | Store | Kept anyway | |---|---|---| | `subscription-runs` | Subscription run history, with each run's notifications | Each subscription's newest run that was not skipped: the baseline its trigger compares the next result count with | | `notifications` | Delivered notifications | | | `anomaly-baselines` | Anomaly detection baselines | The longest configured look-back beyond the window | | `anomaly-events` | Detected anomaly events | The longest configured look-back beyond the window | | `ai-actor-executions` | AI actor executions and conversation turns | An execution a query change or a newer turn still names | | `ai-actor-plans` | AI actor plans | Plans awaiting approval or executing; a plan an execution, a query change or a revision still names | | `migration-runs` | Data migration runs | Queued or running runs, and a run a retry names as its parent | | `mcp-eval-runs` | MCP eval runs and their per-case results | Running evals; the golden cases are never purged | | `execution-log` | The execution log of previews and ad-hoc runs | | What goes with a purged row: a task keeps its row, but its notifications (the runs that raised it) go with their window; a plan that named a deleted AI actor execution has that id cleared; an anomaly event that named a deleted notification keeps no link to it. The content stores are the subscription run history, notifications, the execution log of previews and ad-hoc runs, AI actor executions and conversations, and migration runs. Clearing a row's content empties its SQL text, result snapshot and model output, and reduces its error text to an error class (`schema`, `timeout`, ...), as the write-time lock does. While the content lock is on (`RetainQueryContent` off in the **global** MCP settings, or `Beacon:Mcp:ForceNoContentRetention`), Beacon writes no content to them, and each run also clears the result snapshots they stored before, whatever their age. The content lock reaches these stores only globally: a project's own `RetainQueryContent` override does not, because their rows belong to no single project (a multi-step preview logs no data source, notifications and AI actor turns carry none, and a data source can be linked to several projects, or to others after the row was written). Turn content retention off globally, or set `ForceNoContentRetention`, where any project needs it off for them. The windows are read once at startup. Each step (the content, then each target) gets an equal share of the time left in the run's 50-minute budget; a step whose share runs out stops there (event 9124, the step's name), the run still succeeds, and the next run continues it, so a large first backlog takes several runs and never starves the steps after it. A target that fails is logged (event 9121, the target's name and a failure code) and the others still run, then the job fails. Each purge logs its count (9120, 9122) to the `Beacon.Audit` category. The MCP audit (`Beacon:Mcp:Audit:RetentionDays`), the security audit trail (`Beacon:Audit:RetentionDays`) and learning signals keep their own purges; the security audit is never a retention target. Outside the lock by design, because they are configuration or governance records rather than run history: the security audit trail (its own `Beacon:Audit:RetentionDays`), learning signals (their own window, which the lock caps at `Days`), the query step change history (the record of how a saved query's SQL changed and who, or which AI actor run, changed it; it goes with its query, and the AI actor executions and plans it names are kept for it), the settings change history, and configuration itself: projects, data sources, metadata, queries, subscriptions, tasks, the glossary, golden eval cases and learned patterns. :::note[Upgrading] Nothing is deleted unless you configure it: every window defaults to keep forever, so an upgrade keeps all existing data. While no window is set (`Days`, a target, `ContentDays`, `Beacon:Mcp:Audit:RetentionDays`), the host logs one Information at startup (`RetentionUnconfigured`, 9123) naming these settings. Recommended: `Days: 90`, `Beacon:Mcp:Audit:RetentionDays: 365`, and `ContentDays: 30` for regulated deployments. The sample host expires completed Warp jobs (their records in the `warp` schema, not Beacon data) after 7 days (`JobExpirationTimeout`). ::: ## Deployment locks Settings that bind Admins too and that nothing at runtime can change; only host configuration sets them: | Key | Default | Pins | |---|---|---| | `Beacon:Mcp:ForceReadOnly` | **`true`** | MCP read-only enforcement on, globally and per project; the SQL gate judges every statement read-only whatever it is asked. `false` logs a startup Warning | | `Beacon:Mcp:ForcePiiDetection` | **`true`** | PII detection on. `false` logs a startup Warning | | `Beacon:Mcp:ForceNoContentRetention` | `false` | Content retention off: questions, SQL, result snapshots, model output and free-text errors are never persisted, in MCP and in the content stores outside it (subscription runs, notifications, the execution log, AI actor runs, migration runs; see [Data retention](#data-retention-beaconretention)) | | `Beacon:Mcp:Audit:RequireContent` | `false` | Content retention on, globally and per project, so the MCP audit keeps the question and the SQL of every statement a call ran. Cannot be combined with `ForceNoContentRetention`: with both on the host fails to start. See [Keeping audit content](https://moberghr.github.io/beacon/mcp/observability-and-audit/#keeping-audit-content-the-requirecontent-lock) | | `Beacon:Mcp:ForbidRowDataToLlm` | `false` | Sample-value collection, value grounding and the eval judge off: no value read from a data source goes to the LLM provider | | `Beacon:Mcp:Ceilings:*` | none | Cap the MCP row limit and the SQL and question lengths; the other ceilings clamp recorded settings (see [MCP](#mcp-beaconmcp)) | | `Beacon:Mcp:Audit:Required` | `false` | Withhold an MCP tool result whose audit row could not be written | | `Beacon:Migrations:Enabled` | `false` | Data migration, the only write path | | `Beacon:Retention:Locked` | `false` | No retention window may keep data forever (the host fails to start otherwise), and the learning-signal retention never resolves above `Beacon:Retention:Days`. See [Data retention](#data-retention-beaconretention) | A write that contradicts an MCP lock answers `409` (`/errors/setting-locked`), an Admin's included; the MCP settings page shows the locked settings disabled, and `GET /beacon/api/mcp/settings/locks` lists them (`lockedFields`) and each active lock with the setting it pins (`locks`: `fieldName`, `lockName`). Stored values a lock overrides are kept and listed in one startup warning; they apply again when the lock is lifted. Settings resolve as **lock → project override → global value → built-in default**, ceilings last. Other startup-validated keys (`Beacon:Authorization:*`, `Beacon:Authentication:*`, `Beacon:UserManagement:*`, `Beacon:ApiKeys:*`, `Beacon:Query:*`, `Beacon:Subscriptions:*`, `Beacon:Notifications:*`, `Beacon:EncryptionKey`, `Beacon:DataProtection:*`) have no runtime override either. Of the configuration values, only the base URL and the LLM settings can also be changed at runtime, in [Admin settings](https://moberghr.github.io/beacon/security/admin-settings/). ## Regulated profile (`Beacon:Profile`) `Beacon:Profile: Regulated` turns on a preset of the settings a regulated deployment needs. Unset (or `None`) changes nothing; any other value fails startup. The profile reads the host's configuration keys: a key the host left unset is filled, a key the host set weaker than the profile requires is **never overridden** — the host fails to start with every such key listed in one message. | Key | Under the profile | |---|---| | `Beacon:Mcp:ForceReadOnly`, `Beacon:Mcp:ForcePiiDetection` | Already `true` by default; `false` fails startup | | `Beacon:Mcp:Audit:Required` | Unset: `true`. `false` fails startup | | `Beacon:Mcp:ForbidRowDataToLlm` | Unset: `true` (no sample values, value grounding or eval judge). `false` fails startup | | `Beacon:Mcp:Ceilings:MaxRowLimit`, `StatementTimeoutSeconds`, `MaxResultBytes`, `MaxConcurrentQueriesPerKey` | Unset: the built-in MCP settings defaults (1,000 rows, 30 s, 262,144 bytes, 4 queries per key), so an Admin cannot raise them. A value the host sets is kept | | `Beacon:Retention:Locked` | Unset: `true`, so `Beacon:Retention:Days`, `ContentDays`, every target and `Beacon:Mcp:Audit:RetentionDays` must be configured above zero (see [Data retention](#data-retention-beaconretention)). `false` fails startup. The profile sets no window itself: nothing is deleted that the host did not configure | | `Beacon:RateLimits:*:PermitLimit` (`Login`, `Api`, `Anonymous`, `Mcp`, `Ask`) and `Beacon:RateLimits:Lockout` | Already on by default; a limit of `0`, or a lockout turned off, fails startup (see [Request limits](#request-limits)) | | `Beacon:UserManagement:LocalLogin:Enabled` | Must be set to `false`; never filled, since it ends password sign-in for every account but the break-glass ones (`BreakGlassUserNames` stay allowed) | | `Beacon:Authentication:Oidc:RequireMfa` | With single sign-on on, must be set to `true`; never filled, since your identity provider must send the `amr`/`acr` claims first | ```json { "Beacon": { "Profile": "Regulated", "UserManagement": { "LocalLogin": { "Enabled": false, "BreakGlassUserNames": [ "admin" ] } }, "Authentication": { "Oidc": { "Enabled": true, "RequireMfa": true } }, "Retention": { "Days": 90, "ContentDays": 30 }, "Mcp": { "Audit": { "RetentionDays": 365 } } } } ``` At startup Beacon logs one report of these checks (event 9130, Warning) naming each setting that is weaker than the profile requires, also when no profile is set; under the profile, once every check holds, one Information (9131) names the settings the profile filled. The report carries configuration names only, never a value. LLM concurrency and rate limits are Admin settings stored at runtime (see [AI / LLM](#ai--llm)), outside the profile. ## MCP (`Beacon:Mcp`) Validated at startup. Without the section, `ForceReadOnly` and `ForcePiiDetection` are on, nothing else is locked, and only the built-in length caps apply. ```json { "Beacon": { "Mcp": { "ForceReadOnly": true, "ForcePiiDetection": true, "ForceNoContentRetention": false, "ForbidRowDataToLlm": false, "Ceilings": { "MaxRowLimit": 1000, "MaxSqlChars": 100000, "MaxQuestionChars": 4000, "AskTimeoutSeconds": 120, "AskMaxLlmCalls": 12, "AskSchemaContextChars": 120000 }, "Audit": { "Required": true, "RetentionDays": 400, "RequestIdHeader": "X-Request-Id", "RequireContent": false } } } } ``` | Key | Type | Default | What it does | |---|---|---|---| | `ForceReadOnly`, `ForcePiiDetection`, `ForceNoContentRetention`, `ForbidRowDataToLlm` | bool | `true`, `true`, `false`, `false` | See [Deployment locks](#deployment-locks) | | `Ceilings:MaxRowLimit` | int | none | Caps the row limit the MCP tools apply; must be greater than 0 when set | | `Ceilings:StatementTimeoutSeconds`, `MaxResultBytes`, `MaxConcurrentQueriesPerKey` | int | none | Clamp the MCP settings of the same names, shown on the MCP settings page; must be greater than 0 when set. The MCP tools apply the statement timeout to every statement, bound each tool result to the byte budget, and refuse a call when the caller already runs that many queries | | `Ceilings:LearningSignalRetentionDays` | int | none | Clamps the learning-signal retention (`LearningSignalRetentionDays`, global and per project) on the MCP settings page; greater than 0 when set. Under `Beacon:Retention:Locked` it is at most `Beacon:Retention:Days` | | `Ceilings:MaxExplainCost` | decimal | none | Clamps the MCP `MaxExplainCost` setting, shown on the MCP settings page; greater than 0 when set. The setting is enforced on PostgreSQL data sources: a statement whose planner-estimated total cost (`EXPLAIN`, never run) exceeds it is refused before it runs. Other engines ignore it | | `Ceilings:MaxSqlChars` | int | `100000` (always applies) | Longest SQL text the gate evaluates | | `Ceilings:MaxQuestionChars` | int | `4000` (always applies) | Longest question `ask` accepts | | `Ceilings:AskTimeoutSeconds` | int | `120` (always applies) | Wall-clock time one `ask` call may take, its statements included; past it the call returns an error; 1 to 3600 when set | | `Ceilings:AskMaxLlmCalls` | int | `12` (always applies) | Logical LLM calls one `ask` call may make (a provider's transport retries are not counted); fewer self-consistency candidates and skipped repairs keep it within them, and a call with none left before it generated SQL returns an error; at least 2 when set (the intent classification and the first SQL generation) | | `Ceilings:AskSchemaContextChars` | int | `120000` (always applies) | Longest schema context one `ask` sends to the LLM, in characters (about 30,000 tokens at the default); over it whole table descriptions are left out, least relevant to the question first, and one line inside the cap names them. When the grounding sections leave no room for the most relevant table they are left out first (learned patterns, glossary, verified examples, value matches, join paths), and that table is sent even if it alone is longer than the cap. At least 10000 when set | | `Audit:Required` | bool | `false` | Fail closed: withhold a tool result whose audit row could not be written | | `Audit:RetentionDays` | int | none (keep forever) | Days audit rows are kept; unset or `0` keeps them forever (`365` recommended). The sample host purges daily (`PurgeExpiredMcpAuditLogsJob`); other hosts schedule it themselves | | `Audit:RequestIdHeader` | string | `X-Request-Id` | Upstream correlation header stored with each audit row; empty reads none | | `Audit:RequireContent` | bool | `false` | See [Deployment locks](#deployment-locks) | | `CallerHashKey` | string | derived from `Beacon:EncryptionKey` | See [Encryption key](#encryption-key-configuration-required) | The MCP settings page's own built-in defaults (row limit 1000, PII detection on, content retention on, sample values and value grounding off) are runtime settings, not configuration. See [MCP server](https://moberghr.github.io/beacon/mcp/server/) and [MCP observability and audit](https://moberghr.github.io/beacon/mcp/observability-and-audit/). ### MCP discovery (`Beacon:Mcp:ProtectedResource`) Read at request time. | Key | Type | Default | What it does | |---|---|---|---| | `AuthorizationServers` | string[] | empty | Issuers published in `/.well-known/oauth-protected-resource`; each must equal its server metadata's `issuer` | | `DeriveAuthorizationServerFromJwtIssuer` | bool | `false` | When `AuthorizationServers` is empty, publish the bearer JWT issuer | | `ScopesSupported` | string[] | empty | Scopes published for an external identity provider (for example `api://{id}/mcp.access`) | | `ResourceName` | string | empty | The `resource_name` MCP clients show, also the server card's `name` and the server name at `initialize`; empty publishes `Beacon MCP` (`Beacon` on the card and at `initialize`) | | `ResourceDocumentation` | string | empty | The `resource_documentation` URL, also the server card's `documentation`; empty publishes Beacon's MCP docs. Anything but an absolute https URL logs a startup warning | Without the section, the authority published is `Beacon:Authentication:Oidc:Authority` when SSO is enabled. Startup warns about a derived issuer that is not absolute, and about an external server published without `ScopesSupported`. ## Telemetry | Key | Type | Default | What it does | |---|---|---|---| | `Beacon:Telemetry:CaptureContent` | bool | `false` | Put an MCP tool's input on the current trace span (`beacon.tool.input`), only where the project's content retention allows. Logs and metrics never carry content | | `OTEL_EXPORTER_OTLP_ENDPOINT` | string | unset | Sample host: when set, registers OpenTelemetry tracing and metrics with the OTLP exporter (`/beacon/api/health` and notification calls are not traced) | | `OTEL_SERVICE_NAME` | string | `beacon` | Sample host: the service name | Other standard `OTEL_*` variables are read by the OpenTelemetry SDK itself. The Beacon libraries choose no exporter. ## Embedding (`Beacon:Embed`) Off by default, and a deployment lock: while `Enabled` is false no embed route is mapped (each answers 404) and no key is needed. On, a SecurityAdmin creates embed grants at `POST /beacon/api/embed-grants` (listed at `GET`, revoked at `POST /beacon/api/embed-grants/{id}/revoke`): one project, one saved query and its approved version, the frame origins, an expiry, and every query parameter fixed or locked. A grant's secret is shown once; only its SHA-256 is stored. The host's backend exchanges it for a short-lived token at `POST /beacon/api/embed/tokens` (`Authorization: BeaconEmbedGrant `, body `{ "parameters": { … }, "ttlSeconds": … }`), and Beacon's embed page at `GET /beacon/embed/{publicId}` renders the query with it (`POST /beacon/api/embed/render`, `Authorization: BeaconEmbed `). These three routes are anonymous to Beacon's cookie, API keys and bearer tokens: only the embed credential authenticates them, and every refusal gets the same `{"error":"embed_unavailable"}`. A render acts for the grant's creator (their live access and row entitlement), always masks PII and returns at most the smaller of the project's row limit and `MaxRows`. The embed page is served with a strict Content-Security-Policy whose `frame-ancestors` lists the grant's origins. An MCP gateway host serves no embed route. Validated at startup. The limits count per client address (an IPv6 address by its /64 network): the embed page per address and grant, refusals per address, and the exchange and render per verified grant. Only refusals spend the refusal budget. A render whose token verifies is never refused for it; the exchange and the page, which each cost a lookup, are answered 429 without one once the address has spent it (a valid secret from that address waits for the next minute). **Behind a reverse proxy or load balancer, run `UseForwardedHeaders` with the proxy's address in `KnownProxies` or `KnownNetworks` before `UseBeaconAuthentication`** (the sample host: `Beacon:ForwardedHeaders:Enabled` and `KnownProxies`), and set `Beacon:PublicBaseUrl` when TLS ends at the proxy. Without forwarded headers every viewer shares the proxy's address (one page and refusal budget for all of them), and the render's same-origin check compares the viewer's `https` origin with Beacon's `http` one and refuses every render. The embed routes are `AllowAnonymous`, but ASP.NET Core still authenticates a host's `FallbackPolicy` on them: a fallback policy that names `AuthenticationSchemes` replaces the embed caller and every exchange and render is refused (401). Leave the fallback policy without schemes, or exempt the embed paths from it. | Key | Type | Default | What it does | |---|---|---|---| | `Enabled` | bool | `false` | Maps the embed routes and allows grants to be created | | `SigningKey` | string | Required when `Enabled` | Base64 of at least 32 random bytes (`openssl rand -base64 32`), checked like `Beacon:EncryptionKey`, and never the encryption key itself. Supply as a secret (`Beacon__Embed__SigningKey`) | | `TokenTtlSeconds` | int | `300` | 1 to `MaxTokenTtlSeconds` | | `MaxTokenTtlSeconds` | int | `900` | 1–900 | | `MaxGrantDays` | int | `90` | The longest a grant lives; 1–366 | | `MaxRows` | int | `1000` | 1–100,000 | | `MaxOrigins` | int | `10` | Frame origins per grant; 1–50. Origins are exact `https://host[:port]` (`http://localhost` only in Development) | | `MaxConcurrentRenders` | int | `8` | Embed renders running at once in the process, over every grant; one more is answered 503 at once. 1–256 | | `MaxConcurrentRendersPerGrant` | int | a quarter of `MaxConcurrentRenders`, at least 1 | Renders one grant may run at once, so one grant never holds every slot; one more is answered 503 at once. 1 to `MaxConcurrentRenders` | | `RateLimits:ExchangePerGrantPerMinute` | int | `60` | Exchanges per verified grant; 1–10,000 | | `RateLimits:RenderPerGrantPerMinute` | int | `120` | Renders per verified grant (a valid token); 1–10,000 | | `RateLimits:PagePerClientPerMinute` | int | `60` | Embed page loads per client address and grant; 1–10,000 | | `RateLimits:RefusalsPerIpPerMinute` | int | `30` | Refusals per client address; past it a refusal is answered 429 and not audited, and the exchange and page are 429 before any lookup. 1–10,000 | ## Embed tokens (`Beacon:EmbedToken`, obsolete) Read by `IEmbedTokenService` (obsolete, removed in 6.0), a host-facing service that mints and validates HS256 tokens for embeddable integrations; no Beacon route uses it, and the embed routes never accept its tokens. Validated at first use. | Key | Type | Default | What it does | |---|---|---|---| | `SigningKey` | string | Required | Required; at least 32 bytes (UTF-8). Supply as a secret | | `Issuer` | string | `beacon` | Must not be blank | | `DefaultTtlSeconds` | int | `300` | Greater than 0, at most `MaxTtlSeconds` | | `MaxTtlSeconds` | int | `900` | Greater than 0 | ## Scheduling Beacon does not bundle a job runner; recurring and background work goes through `IBeaconScheduler`, registered with `options.AddBeaconScheduler()` (required). The sample host implements it on Moberg.Warp. ```csharp namespace Beacon.Core.Worker; public interface IBeaconScheduler { Task AddOrUpdate(int subscriptionId, string subscriptionName, string cron); Task Remove(int subscriptionId, string subscriptionName); // Optional: default implementations throw NotImplementedException. Task AddOrUpdateDataQualityJob(int contractId, string contractName, string cron); Task RemoveDataQualityJob(int contractId, string contractName); Task EnqueueProjectDocumentation(int projectId, int userId, string? notifyUserId); Task EnqueueAiActorThinkCycle(int actorId, int subscriptionId); Task EnqueueMcpEval(int runId); } ``` Beacon calls `AddOrUpdate` when a subscription is created or its cron changes and `Remove` when it is deleted or disabled; your implementation maps them onto recurring jobs that call `IJobService.ExecuteQuery(subscriptionId)`. The `Enqueue…` methods return the job id the UI correlates `JobStatusChanged` events by. A host that runs Beacon's jobs on its own job host must mark each job's DI scope as system work (`ISystemWorkScope.MarkAsSystemWork()`). See [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/). ## Data-source connectors Register a connector per engine on the builder: `.AddPostgreSqlConnector()`, `.AddSqlServerConnector()`, `.AddMySqlConnector()`, `.AddBigQueryConnector()`, `.AddSnowflakeConnector()`, `.AddDatabricksConnector()`, `.AddAzureSynapseConnector()`, `.AddCloudWatchConnector()`, `.AddApiConnector()`. Data-source connections are entered in the UI and encrypted at rest. Give every data source Beacon reads a read-only login. On PostgreSQL and MySQL, Beacon also runs each read in a read-only transaction. See [Data-source connectors](https://moberghr.github.io/beacon/integrate/connectors/) and [Data sources](https://moberghr.github.io/beacon/features/data-sources/). ### Warehouse cost limits Each BigQuery query job is sent with `MaximumBytesBilled`: the data source's own `MaximumBytesBilled`, or 10 GiB (10,737,418,240 bytes) when it sets none. A job that would bill more fails without a charge, with a message naming the limit. The per-source value is a number of bytes in the data source's connection JSON, next to `ProjectId` and `ServiceAccountJson` (for example `"MaximumBytesBilled": 2147483648`); the **New data source** dialog does not ask for it, so set it by replacing the connection with `PUT /beacon/api/data-sources/{id}` (with `confirmTargetChange: true`, see [Data sources](https://moberghr.github.io/beacon/features/data-sources/)). The deployment can lower every source's limit, bound from `Beacon:DataSources:BigQuery`: | Key | Type | Default | What it does | |---|---|---|---| | `MaxBytesBilled` | long | none | The most bytes one BigQuery query job may bill; a larger source limit is lowered to it. At least 1 when set, validated at startup | ```json { "Beacon": { "DataSources": { "BigQuery": { "MaxBytesBilled": 5368709120 } } } } ``` Not configurable, on every deployment: - A BigQuery, Databricks or CloudWatch data source's stored `QueryTimeoutSeconds` is clamped to 900 seconds (0 or less means 300). A BigQuery job and a Databricks statement are cancelled on the warehouse when the timeout passes or the caller cancels. - Databricks results are limited to 16 MiB (`byte_limit`); a larger result fails rather than being cut short. A BigQuery or Databricks result read without a row cap fails above 10,000 rows (Beacon's own reads always carry one). - A REST API data source's `TimeoutSeconds` is clamped to 1–120 seconds and one response may hold at most 10 MB. Fetching the OpenAPI description for a metadata import takes at most 30 seconds and 50 MB. ## Logging ```json { "Logging": { "LogLevel": { "Default": "Information", "Microsoft.AspNetCore": "Warning", "Microsoft.EntityFrameworkCore": "Warning" } } } ``` Beacon logs identifiers and counts, never query text, connection strings, row data or tokens. Security log events are structured log events 7101 to 7123; route them to your log pipeline, together with the `Beacon.Audit` category, which carries the MCP audit events and a copy of every committed [security audit](https://moberghr.github.io/beacon/security/audit-trail/#shipping-to-a-siem) event (9104 to 9114). ## Complete example ```json { "ConnectionStrings": { "BeaconContext": "Host=db;Database=beacon;Username=beacon;Pooling=true;MaxPoolSize=50" }, "Beacon": { "BaseUrl": "https://beacon.example.com/beacon", "PublicBaseUrl": "https://beacon.example.com", "DataProtection": { "ApplicationName": "beacon-production" }, "Authentication": { "Session": { "RevalidateMinutes": 5, "AbsoluteLifetimeHours": 12 }, "Oidc": { "Enabled": true, "Authority": "https://login.microsoftonline.com/{tenant-id}/v2.0", "ClientId": "{client-id}", "AllowedTenants": [ "{tenant-id}" ] } }, "UserManagement": { "LocalLogin": { "Enabled": false, "BreakGlassUserNames": [ "superadmin" ] } }, "Authorization": { "RequireProjectMembership": true, "SeparationOfDuties": { "Enabled": true } }, "ApiKeys": { "MaxLifetimeDays": 180 }, "Mcp": { "Audit": { "Required": true, "RetentionDays": 400 } }, "ForwardedHeaders": { "Enabled": true, "KnownProxies": [ "10.0.0.5" ] }, "LLM": { "Provider": "OpenAI", "Model": "" } }, "Logging": { "LogLevel": { "Default": "Information" } } } ``` Supply `Beacon__EncryptionKey`, `Beacon__Authentication__Oidc__ClientSecret`, `Beacon__UserManagement__SetupToken`, `Beacon__LLM__ApiKey` and the database password as environment variables or from a secrets provider. ## Next steps - [Host wiring](https://moberghr.github.io/beacon/integrate/host-setup/): registration and middleware in context - [Production checklist](https://moberghr.github.io/beacon/integrate/production/) - [Security model](https://moberghr.github.io/beacon/security/) --- # Routes & endpoints > Every URL Beacon serves: UI pages under /beacon, anonymous routes, REST API groups under /beacon/api, the SignalR hub, the MCP server and discovery documents, with the paging contract and error responses. Source: https://moberghr.github.io/beacon/reference/routes/ Beacon serves everything under `/beacon`. The site root `/` belongs to the host application. Paths below are absolute; replace the origin with your host's. | Prefix | What | |---|---| | `/beacon` | The React UI (`app.MapBeaconUi()`). | | `/beacon/api` | REST API and SignalR hub (`app.MapBeaconApi()`, `MapLoginEndpoints`, `MapSetupEndpoints`). | | `/beacon/mcp` | MCP server (`app.MapBeaconMcp()`). | | `/.well-known/…` | MCP discovery documents (`app.MapMcpDiscovery()`). | ## UI pages The UI is a single-page app with the router mounted at `/beacon`. Every path below is under `/beacon`, for example `/beacon/queries`. Signed-out visitors are sent to `/beacon/login`. On a new installation, open `/beacon/setup` to create the first admin. "Project access" means a user with a Viewer or Editor role, or an Admin. A user who holds only functional permissions does not see those pages. The UI hides what a user cannot use; the API enforces it. | Path | Page | Who sees it | |---|---|---| | `/login`, `/logout` | Sign in, sign out | Anyone | | `/setup` | First-run setup: create the super admin | Anyone, while no admin exists | | `/error` | Error page | Anyone | | `/`, `/home` | Home: KPIs, trends, activity, uptime | Everyone signed in | | `/about` | About Beacon | Everyone signed in | | `/settings` | Your settings and password | Everyone signed in | | `/control-tower` | Control Tower: health of subscriptions and contracts | Project access | | `/data-sources`, `/data-sources/:id` | Data sources list and detail | List: project access or DataSourceAdmin. Detail: project access | | `/data-sources/:id/relationships` | Schema relationships and schema health | Project access | | `/projects`, `/projects/new`, `/projects/:id` | Projects: members, linked data sources, documentation | Project access | | `/data-quality`, `/data-quality/:id` | Data contracts overview and detail | Project access | | `/migration-jobs`, `/migration-jobs/new`, `/migration-jobs/:id`, `/migration-history` | Data migration | MigrationOperator or Admin, only when `Beacon:Migrations:Enabled` is on | | `/ai-actors`, `/ai-actors/:id`, `/ai-actors/:id/edit` | AI actors | Project access | | `/queries`, `/queries/new`, `/queries/:id`, `/queries/:id/edit` | Queries list, new query, detail, editor | Project access | | `/queries/:id/versions`, `/queries/:id/versions/:versionId` | Query versions | Project access | | `/subscriptions`, `/subscriptions/:id` | Subscriptions list and detail | Project access | | `/notifications`, `/notifications/:id` | Notification history and detail | Project access | | `/recipients` | Recipients | Project access (changes: SecurityAdmin or Admin) | | `/tasks`, `/tasks/:id` | Tasks | Project access | | `/api-keys` | API keys | Project access, Auditor or UserAdmin | | `/mcp-settings` | MCP guardrail settings | Auditor (read) or SecurityAdmin | | `/mcp-playground` | Try the MCP tools in the browser | Project access | | `/mcp-learning` | Learned patterns and documentation patches | Project access | | `/approvals`, `/approvals/:id`, `/approvals/changes/:id` | Approvals of query changes and other changes | Approver or Admin | | `/users` | User management | UserAdmin or Admin | | `/admin-settings` | Admin settings, LLM provider | Auditor (read) or SecurityAdmin | Any other path under `/beacon` redirects to `/beacon/home`. ## Anonymous routes These answer without a session. Everything else requires one, an API key or a bearer token. | Method and path | What | Notes | |---|---|---| | `GET /beacon/api/health` | Health check | `200 {"status":"ok"}` when the metadata database answers within 5 seconds, otherwise `503` problem. | | `GET /beacon/api/auth/me` | The current session, if any | | | `GET /beacon/api/auth/sso` | Whether SSO is offered, for the login page | | | `GET /beacon/api/csrf` | Antiforgery token for the UI | `400` for callers authenticated by an `Authorization` header. | | `POST /beacon/api/auth/login` | Password sign-in | Mapped only while password sign-in is on or a break-glass account is configured. Needs the antiforgery token (`X-XSRF-TOKEN`), `400` without it. 10 attempts per minute per client IP, then `429` with `Retry-After`. | | `POST /beacon/api/auth/logout` | Sign out; `?everywhere=true` signs out every session of the user | | | `GET /beacon/api/auth/sso/challenge` | Start the OIDC sign-in | | | `GET /beacon/api/setup/status` | Whether first-run setup is open | Mapped when user management is on. | | `POST /beacon/api/setup/superadmin` | Create the super admin with the setup token | Throttled per IP. `403` wrong token, `409` lost a race, `400` already set up. | | `GET /beacon/api/setup/roles` | Roles, for the setup page | Anonymous only while setup is open. | | `GET /.well-known/oauth-protected-resource` | MCP protected-resource metadata (RFC 9728) | Also at `/.well-known/oauth-protected-resource/beacon/mcp`. | | `GET /.well-known/mcp/server-card.json` | MCP server card | | | `/signin-oidc` | OIDC callback | When OIDC is enabled. Path set by `CallbackPath`. | | `GET /signout-oidc` | Front-channel sign-out | Only with `EnableFrontChannelLogout`. | Until the first admin exists, the anonymous API routes above answer with a redirect to `/beacon/setup`. Only the setup routes, the health check and the MCP discovery documents answer normally. ## REST API All groups are under `/beacon/api` and need a signed-in session, an API key (`Authorization: Bearer sk-sem_…`) or a bearer JWT. For callers with a scope (API keys, mapped MCP callers), every request other than `GET`, `HEAD` or `OPTIONS`, and a few marked reads, need the **Execute** scope. Cookie sessions send the antiforgery header `X-XSRF-TOKEN` on state-changing requests. Functional permissions named below are held by Admins too. A by-id request on a resource in a project you cannot read answers `404`; one you can read but may not act on answers `403`. | Prefix | Covers | Notable rules | |---|---|---| | `/auth/me`, `/auth/sso`, `/auth/permissions` | Current user, SSO config, current permissions | `permissions` returns `canRead`, `canWrite`, `isAdmin`, `permissions`, `migrationsEnabled`, `build` (`version`, `commit`). | | `/home` | Home page: trends, activity, task summary, uptime, migration summary | `migration-summary` admits MigrationOperator. | | `/projects` | Projects, linked data sources, members, access mode, repositories and their scans (in the request, or as a background task), documentation, imported documents, MCP context | Members: project admins, UserAdmin. Access mode: UserAdmin (opening a restricted project: Admin). Unlinking a data source: Admin. Repository token: SecurityAdmin. | | `/data-sources` | Data sources, connection test, metadata, metadata refresh | Create, test and delete: DataSourceAdmin. `GET /{id}/metadata` needs Execute scope. `PUT /{id}/migration-writes`: Admin. Forced refresh is rate limited per data source (`429`). | | `/data-sources/{id}/relationships`, `/data-sources/{id}/schema-health` | Schema relationships: list, add, edit, verify, delete, discover; schema health | Verifying needs the Curate action. | | `/queries` | Queries: list, detail, create, update, lock, change history, preview, step preview, publish as MCP tool | No delete route. Previews need Execute scope. `PUT /{id}/mcp-tool`: SecurityAdmin. | | `/queries/{id}/versions`, `/query-versions` | Versions: list, detail, diff, restore | | | `/query-folders`, `/queries/{id}/folder` | Query folders and moving a query between them | | | `/subscriptions` | Subscriptions: list, create, detail, delete, reactivate, SLA, test run, recipients, anomaly chart, owner | Changes need Execute scope. `PUT /{id}/owner`: UserAdmin. No route changes the schedule. | | `/recipients` | Recipients | Create, update, delete: SecurityAdmin. | | `/notifications` | Notification history and detail | Filtered to the caller's projects. | | `/tasks` | Tasks: list, detail, resolve, assign, snooze, priority, watch, comments, executions, result history | | | `/control-tower` | Statistics, health grid, subscription detail | Filtered to the caller's projects. | | `/data-quality` | Overview, data contracts, evaluations, evaluate now, owner | Changes need Execute scope. `PUT /contracts/{id}/owner`: UserAdmin. Custom SQL rules: Admin. | | `/approvals` | Pending query approvals, change approvals (`/approvals/changes`), approve, reject | Reading admits Approver; approve and reject require Approver. | | `/migrations` | Migration jobs and executions | Mapped only when `Beacon:Migrations:Enabled` is on. MigrationOperator. | | `/ai-actors` | AI actors, think, pause, resume, refine, plans | `PUT /{id}/owner`: UserAdmin. | | `/mcp` | MCP settings and locks, project settings, learned patterns, documentation patches, learning stats, playground tools, audit log | Settings: read Auditor or SecurityAdmin, write SecurityAdmin. `POST /tools/run` needs Execute scope. `GET /audit`: Auditor. | | `/eval` | Golden examples, candidates, feedback, grounding re-check, eval runs | Admin. | | `/glossary` | Business glossary | Admin. | | `/api-keys` | Your API keys; `/api-keys/admin` for all keys | Create from a signed-in session only. `GET /admin`: Auditor or UserAdmin. `DELETE /admin/{id}`: UserAdmin. | | `/users` | Users, roles, enable/disable, sign out everywhere | UserAdmin. | | `/user-settings` | Your settings, change your password | | | `/admin-settings` | Runtime settings, LLM test | Read: Auditor or SecurityAdmin. Write and test: SecurityAdmin (LLM connection and base URL: Admin). | The OpenAPI document at `/openapi/v1.json` lists every operation with its schema (see [Host-mapped routes](#host-mapped-routes)). ### Paging, sorting and filtering Every list endpoint takes the same query parameters: | Parameter | Default | Rules | |---|---|---| | `page` | `0` | Zero-based. Negative values count as `0`. | | `pageSize` | `20` | Clamped to 1–200, never rejected. | | `sort` | the list's own order | Comma-separated property names, `-` for descending, for example `sort=-createdTime,name`. Unknown names are ignored. | A paged response is `{ "items": [...], "totalCount": n, "pageCount": n }`. Lists are filtered to the projects the caller can read in SQL, before paging, so counts never include rows you cannot see. Five lists predate paging and keep their original route and array response, and are deprecated in the OpenAPI document. They return every row up to a 10,000 safety cap; when the cap cuts, the response carries `X-Beacon-Truncated: true` and Beacon logs a warning. Each has a paged twin, which new callers should use: | Deprecated unpaged route | Paged route | |---|---| | `GET /glossary` | `GET /glossary/paged` (also `search` on the term) | | `GET /eval/golden` | `GET /eval/golden/paged` (also `id`) | | `GET /eval/runs/{runId}/results` | `GET /eval/runs/{runId}/results/paged` | | `GET /query-folders` | `GET /query-folders/paged` (also `search` on the name or path) | | `GET /data-quality/overview` | `GET /data-quality/overview/paged` (one item per data source, each with its 20 lowest-scoring tables; the unpaged overview lists the same 20 per source, with counts and averages over all of a source's tables) | ### Errors | Situation | Response | |---|---| | No session, key or token on an API route | `401`, `application/problem+json`: `{"type":"about:blank","title":"Unauthorized","status":401}`. API routes never redirect to the login page. | | Signed in, but the role, scope or permission does not allow the action | `403` problem. | | A resource that does not exist, or that is in a project you cannot read | `404` problem. Beacon does not reveal whether it exists. | | Readable, but the action is not granted in that project | `403` problem. | | A conflicting change (a stale approval hash, a membership race, a locked setting) | `409` problem. A locked setting uses type `/errors/setting-locked`. | | Too many requests (sign-in, setup, forced metadata refresh) | `429`. | The sample host also maps unhandled exceptions under `/beacon/api` to problem responses: `400` for business-rule errors (types `/errors/invalid-operation`, `/errors/beacon`) and `500` (`/errors/internal`) otherwise. That mapping is sample code; an embedding host supplies its own. ## Realtime hub | Path | Protocol | Who | |---|---|---| | `/beacon/api/hub` | SignalR | Signed-in users who can read project data, or who hold a functional permission whose pages use hub events; others get `403`. The connection is closed when the session's next check is due, and the client reconnects. | Events are sent only to the user they concern: | Event | Payload | When | |---|---|---| | `JobStatusChanged` | `jobId`, `state`, `timestamp` | A background job the user started changed state. Published by the host's job runner (the sample's Warp pipeline does). | | `ApprovalUpdated` | `approvalId`, `status` | An approval request was approved or rejected. | A host turns the hub off with `AddBeaconApiServices(x => x.Realtime = false)`. ## MCP server | Path | What | |---|---| | `/beacon/mcp` | MCP over Streamable HTTP. Needs the Execute scope: an API key (`Authorization: Bearer sk-sem_…`) or a mapped Entra bearer token. A `401` carries `WWW-Authenticate: Bearer resource_metadata="…/.well-known/oauth-protected-resource"`. | The discovery documents are listed under [Anonymous routes](#anonymous-routes). Set `Beacon:PublicBaseUrl` so they advertise your public origin. See [MCP server](https://moberghr.github.io/beacon/mcp/server/). ## Host-mapped routes These come from the host, not from the Beacon packages. The sample host maps them; your host decides for itself. | Path | What | |---|---| | `/openapi/v1.json` | OpenAPI document of the REST API (`app.MapBeaconOpenApi()`). Open in Development (the UI's typed client is generated from it there); elsewhere it needs a signed-in Beacon caller unless `Beacon:OpenApi:Anonymous` is `true`. A gated document needs `/openapi` in `Beacon:Authentication:AdditionalPaths`. | | `/warp` | Warp job dashboard, sample host only. Signed-in Admins. |