# Host Docs

> Ship the host application's markdown documentation with it and import it into a Beacon project. The documents are searchable by MCP agents with or without embeddings, readable through get_documentation, and summarised in an AGENTS.md project brief.

Source: https://moberghr.github.io/beacon/features/host-docs/

A host application often keeps its own documentation next to the code: a `docs/wiki/` folder of generated markdown
pages, runbooks, conventions. `ExposeDocs` imports those documents into a Beacon project at startup
so MCP agents can `search` them and read them with `get_documentation`, alongside the host's data
([Host DbContext](https://moberghr.github.io/beacon/features/host-dbcontext/)). It works with `UseAI = false`: no LLM and no embedder is needed.

## Quick start

```csharp
var beacon = builder.Services.AddBeaconServices(builder.Configuration, o => { /* your options */ });

beacon.ExposeDbContext<HostContext>(o => o
    .ReadOnlyConnection("BeaconReadOnly")
    .AllowTables("Loan", "LoanInstallment", "Customer"));

beacon.ExposeDocs(o =>
{
    o.ProjectName = "HostContext";             // the project ExposeDbContext uses (its Name/ProjectName)
    o.FromDirectory("docs/wiki")               // relative to ContentRootPath; default glob **/*.md
     .Exclude("drafts/**");
});

beacon.UseSqlServer();                         // ends the builder chain: call ExposeDocs first

var app = builder.Build();
// ... apply migrations ...
await app.Services.SyncBeaconHostAsync();      // syncs ExposeDbContext data sources, then ExposeDocs documents
```

`SyncBeaconHostAsync` is the single startup entrypoint for everything a host declares. It runs the
`ExposeDbContext` sync first (exactly what `SyncBeaconHostDataSourcesAsync` does; that call still works on its
own) and then imports the documents. It is a plain startup call, not a hosted service.

Both calls are **safe to run from several replicas at once**: the whole sync holds one lock in Beacon's own database
(PostgreSQL `pg_try_advisory_lock` on a fixed key, SQL Server `sp_getapplock` with a session owner), so replicas
starting together run it one after another instead of racing each other's inserts. A replica waits up to 5 minutes
for the lock. A unique-key race that still happens (a row created outside the lock) re-runs the idempotent sync once
instead of failing startup. Other providers run the sync unlocked and log a warning. A host that coordinates startup
another way can register its own `IHostSyncLock`.

### The host project

`ExposeDbContext`, `ExposeDocs` and `AddHostEndpointTools` all resolve their `ProjectName` through one shared
resolver, so they always land on the same project. The host project is identified by a **host key**
(`Project.HostManagedKey = host:{ProjectName, trimmed and lower-cased}`), never by its display name, which is neither
unique nor protected:

- On first sync the host project is created with the key. If a project that is **not** host-managed already uses the
  name (for example one a Beacon user created by hand), it is left untouched and the host project is created as
  `"{ProjectName} (host)"` (then `"(host 2)"`, …), with a warning in the log. A user project with the same name never
  receives the host database, documents or endpoint tools.
- Renaming the host project in the UI is fine: it is still found by its key.
- An archived host project is restored, because the host still declares it.
- Changing `ProjectName` creates (or claims) the project for the new name, and the host data source is **unlinked
  from every other project**, including the project of the old name, so it loses access to the host database.

## Shipping the documents

The docs are not part of the build output by default, so the host has to ship them. Either copy them next to the
app:

```xml
<!-- YourHost.csproj -->
<ItemGroup>
  <Content Include="..\docs\wiki\**\*.md" CopyToOutputDirectory="PreserveNewest" LinkBase="docs/wiki" />
</ItemGroup>
```

```csharp
o.FromDirectory("docs/wiki");
```

`FromDirectory` resolves a relative path against `IHostEnvironment.ContentRootPath`. Make sure that is the
output directory in your deployment (it is when the app is started from its own folder), or pass an absolute
path such as `Path.Combine(AppContext.BaseDirectory, "docs/wiki")`.

Or embed them in the assembly:

```xml
<ItemGroup>
  <EmbeddedResource Include="..\docs\wiki\**\*.md" LogicalName="docs/wiki/%(RecursiveDir)%(Filename)%(Extension)" />
</ItemGroup>
```

```csharp
o.FromEmbeddedResources(typeof(Program).Assembly, "docs/wiki/");
```

The part of the resource name after the prefix is the document path (backslashes become `/`). Without a
`LogicalName`, MSBuild mangles folder names (`40-web` becomes `_40_web`) and joins them with dots, so set one.

## Options

| Option | Meaning |
|---|---|
| `ProjectName` | **Required.** The host project the documents attach to, found by its host key (see [The host project](#the-host-project)); created when missing. |
| `FromDirectory(path, params globs)` | Files under `path` (relative to the content root, or absolute) matching the globs. Default glob `**/*.md`. |
| `FromEmbeddedResources(assembly, prefix)` | Manifest resources whose name starts with `prefix`. |
| `Exclude(params globs)` | Skips documents whose relative path matches (applies to every source of this call). |
| `MaxDocumentBytes` | Per-document cap, default 512 KB. Larger files are skipped with a warning that names only the path. |

`ExposeDocs` may be called more than once, for several folders or for several projects. Only `.md`,
`.markdown` and `.txt` files are ingested, whatever the glob matches. Globs must be relative and may not contain
`..`; a file that resolves outside the configured directory, and any symbolic link inside it (file or folder), is
rejected. A configured directory that does not exist fails startup, so a broken deployment cannot silently archive
every imported document.

## What the sync does

Each document is stored as a `ProjectImportedDocument`, keyed by project, source (`docs:dir:docs/wiki`,
`docs:res:{Assembly}:{prefix}`) and relative path.

- **Title**: the frontmatter `title`, else the first `# ` heading (outside code fences), else the file name.
- **Frontmatter**: a leading `---` YAML block is stripped from the body and kept as JSON (flat keys, quoted
  strings, `[a, b]` and `- item` lists, such as a `sources:` list).
- **Idempotent**: a document whose content hash is unchanged is left alone; nothing is written when nothing
  changed. A changed document is updated in place, a removed file is archived, and a file that comes back is
  un-archived.
- **Case-insensitive paths**: source and path are matched ignoring letter case, as SQL Server's unique index
  compares them. Renaming `Guide.md` to `guide.md` updates the existing row (it takes the new casing). Two files
  whose paths differ only by case (possible on a case-sensitive file system or in embedded resources) cannot both be
  stored: the first is imported and the other is skipped with a warning that names its path.
- **Chunked for search**: new and changed documents are split into sentence-window chunks (the same chunker and
  `DocChunkWindowSentences` / `DocChunkOverlapSentences` settings as generated documentation). Keyword search
  works on them immediately. When the local embedder and semantic retrieval are enabled, the sync then embeds the
  new chunks through the doc-chunk re-index, and the recurring `mcp-docchunk-reindex` job keeps them current. An
  embedding failure is logged and does not fail startup.

The documents belong to one project and are only ever visible to callers authorized for that project.

## On the MCP surface

- **`get_documentation`**: the project-level response (no arguments) ends with an *Imported documents* listing
  (title + path). `document: "<path>"` (or an exact title) returns one document in full; a title shared by several
  documents is an error that names their paths.
- **`search`**: imported documents match on title, path or chunk text and appear as
  `- **[IMPORTED_DOC]** \`path\` (title)`, followed by ` -- snippet` when there is one, with or without embeddings.
- **`get_context` with `format: "agents_md"`**: a deterministic *project brief* in markdown to drop into an agent
  workspace's `AGENTS.md`: the project description, data sources (host-managed ones marked read-only), up to 30 key
  tables (best-described and most-related first), masked columns, up to 30 glossary terms, the imported document
  titles, and how to use Beacon's tools (`get_query_context` → `dry_run` → `query`, read-only, masked columns stay
  masked). Built from the catalog only, with no LLM call, and audited like every tool call.

The REST API exposes the same documents: `GET /beacon/api/projects/{id}/imported-documents` (listing) and
`GET /beacon/api/projects/{id}/imported-documents/{documentId}` (one document with its content). Both need Read in
the project under [project access](https://moberghr.github.io/beacon/features/authorization/#project-access), for sessions and API keys alike; a
project the caller cannot read answers `404`.

## Example: data and docs in one project

```csharp
beacon.ExposeDbContext<HostContext>(o =>
{
    o.Name = "BackOffice";
    o.ProjectName = "Back Office";
    o.ReadOnlyConnection("BeaconReadOnly").AllowTables("Loan", "LoanInstallment", "Merchant", "Customer");
});

beacon.ExposeDocs(o =>
{
    o.ProjectName = "Back Office";
    o.FromDirectory("docs/wiki");
});
```

With the `Content Include` item above, the pages land in `docs/wiki/` next to the host's assembly. On each start only
the pages that changed since the last deployment are re-chunked.