# AI providers

> Turn on Beacon's AI layer: UseAI and AddBeaconAI, the four LLM providers and their settings, the request queue, runtime switching, local embeddings for semantic retrieval, and what data reaches the provider.

Source: https://moberghr.github.io/beacon/integrate/ai/

Beacon's AI layer powers natural-language questions (`ask`), project documentation, grounding and the learning loop, and AI actors. It is optional, except that the MCP package depends on the AI package, so an MCP host always registers it.

## Turning it on

```csharp
builder.Services.AddBeaconServices(builder.Configuration, options =>
{
    options.UseAI = true;
    // …
});

builder.Services.AddBeaconAI(builder.Configuration);   // binds Beacon:LLM, registers providers and the queue
```

Set both. `UseAI = false` registers stand-ins for the AI-only services; `UseAI = true` without `AddBeaconAI` leaves those services unregistered.

## Choosing a provider

Configure the provider in host configuration (`Beacon:LLM`). At startup Beacon builds the provider from that section. An Admin can also change the provider in **Admin settings** (`/beacon/admin-settings`): saving swaps the active provider immediately, without a restart, and keeps the previous one if the new configuration cannot be built.

:::note
The provider is built from `Beacon:LLM` each time the process starts. Keep that section in step with what you choose in Admin settings, or save the Admin settings again after a restart.
:::

| `Provider` | Required | Notes |
|---|---|---|
| `OpenAI` | `ApiKey`, `Model` | Calls the public OpenAI API. |
| `Claude` (shown as **Anthropic** in the UI) | `ApiKey`, `Model` | Calls `https://api.anthropic.com`. |
| `AzureOpenAI` | `Endpoint`, `ApiKey`, `Model` (the **deployment name**) | |
| `Bedrock` | `Region`, `Model` (model id) | `ApiKey` empty → the default AWS credential chain (instance role, environment); `accessKey:secretKey` → explicit keys, plus `SessionToken` for temporary credentials. |

```json
{
  "Beacon": {
    "LLM": {
      "Provider": "Claude",
      "ApiKey": "<from a secret store>",
      "Model": "<model id>",
      "FastModel": "<smaller model id>",
      "Limits": { "MaxConcurrentRequests": 20 }
    }
  }
}
```

```json
{
  "Beacon": {
    "LLM": {
      "Provider": "Bedrock",
      "Region": "eu-central-1",
      "Model": "<bedrock model id>"
    }
  }
}
```

`Model` and `FastModel` are free text. Use whatever model ids your provider offers. `Model` does the work (SQL generation, documentation, actors); `FastModel` is stored with the configuration for a smaller model.

### The request queue

Every LLM call goes through one queue that allows at most `Limits:MaxConcurrentRequests` calls at a time (default 50) and makes up to three attempts for a retryable failure, with exponential backoff. The concurrency limit is read at startup. Token, request and budget figures (`TokensPerMinute`, `RequestsPerMinute`, `MonthlyBudget`) are recorded and shown in Admin settings; use your provider's own quotas and spend limits for hard caps.

## Local embeddings (optional)

Semantic retrieval (finding the right tables, examples and documentation chunks for a question) can use a local ONNX embedding model. Nothing leaves your process.

```json
{
  "Beacon": {
    "Embeddings": {
      "Enabled": true,
      "ModelPath": "/models/bge-small-en-v1.5/model.onnx",
      "TokenizerPath": "/models/bge-small-en-v1.5/vocab.txt"
    }
  }
}
```

- Model: **bge-small-en-v1.5** (384 dimensions, 512 tokens). Beacon does not ship the model files; download them and mount them read-only.
- Embeddings are used only when `Enabled` is `true` **and** both files exist. Otherwise Beacon falls back to keyword retrieval.
- On PostgreSQL vectors live in a `pgvector` column with an HNSW index; on SQL Server they are compared in memory.
- Schedule the re-index jobs (`mcp-embedding-reindex`, `mcp-docchunk-reindex`); see [Background jobs](https://moberghr.github.io/beacon/integrate/background-jobs/#recurring-maintenance-jobs).

## What reaches the LLM provider

By default Beacon sends **schema and documentation, not row values**:

| Data | Sent? |
|---|---|
| Table and column names, types, relationships, documentation, glossary, verified examples | Yes (this is the grounding) |
| The user's question | Yes |
| Sample values from tables | Only with the per-project setting **sample-value collection** (off by default); never from PII or secret-like columns |
| Values probed to ground a question (value grounding) | Only with **value grounding** on (off by default) |
| Query results for an LLM judge in evaluations | Only with **eval judge** on (off by default) |
| Database error text | Redacted: value-carrying errors are reduced to their class |

Set the deployment lock `Beacon:Mcp:ForbidRowDataToLlm: true` to pin all three switches off for every project. Every LLM request that carries row-derived data is logged as event **7123 `RowDataSentToLlm`** with caller, project, data source ids and categories, never the values.

## Features that use AI

| Feature | Needs |
|---|---|
| MCP `ask`, `get_query_context` | AI on; a provider for `ask` |
| Project documentation | AI on, a provider, `EnqueueProjectDocumentation` in your scheduler |
| Learning loop, evaluations | AI on; scheduled maintenance jobs; `EnqueueMcpEval` for eval runs |
| AI actors | AI on, a provider, `EnqueueAiActorThinkCycle` in your scheduler |

The MCP `query`, `dry_run`, `search`, `get_context` and `get_documentation` tools work without an LLM provider.

More: [AI integration](https://moberghr.github.io/beacon/features/ai-integration/) · [Knowledge base & grounding](https://moberghr.github.io/beacon/features/knowledge-base/) · [AI actors](https://moberghr.github.io/beacon/features/ai-actors/).