Data Sources

Data Sources represent database connections that you want to monitor with Beacon.

Purpose

Data Sources allow you to:

• Connect to multiple databases and services across different servers
• Organize queries by database or application
• Support nine connectors: PostgreSQL, SQL Server, MySQL, Google BigQuery, Snowflake, Databricks, Azure Synapse, AWS CloudWatch, and a generic REST API
• Reuse connections across multiple queries
• Manage credentials securely (connection strings are encrypted at rest with AES-256 via the required Beacon:EncryptionKey)

Use Cases

• Application Database Monitoring: Create a data source for each application’s database
• Multi-Database Reporting: Connect to different databases for consolidated reporting
• Environment Separation: Separate data sources for dev, staging, and production
• Multi-Tenant Monitoring: One data source per tenant database

Creating a Data Source

Step 1: Navigate to Data Sources

1. Log in to Beacon at the React UI (/login)
2. Click Data Sources in the left navigation menu (/data-sources)
3. Click Create New Data Source

Step 2: Fill Data Source Details

Field	Description	Required	Example
Name	Descriptive data source name	Yes	Production Database
Description	Purpose of this data source	No	Main application database monitoring
Database Type	Connector engine	Yes	PostgreSQL, SQL Server, MySQL, Google BigQuery, Snowflake, Databricks, Azure Synapse, AWS CloudWatch, or REST API
Connection String	Connection details	Yes	See examples below

Encrypted at rest

Connection strings are never stored in plaintext. They are encrypted with AES-256 using the required Beacon:EncryptionKey and decrypted only when a query runs.

Step 3: Configure Connection String

PostgreSQL:

Host=prod-db.company.com;Database=myapp;Username=readonly;Password=secretpass

SQL Server / Azure Synapse:

Server=sql-server.company.com;Database=myapp;User Id=readonly;Password=secretpass;TrustServerCertificate=True

MySQL:

Server=mysql-server.company.com;Database=myapp;Uid=readonly;Pwd=secretpass

Other connectors (Google BigQuery, Snowflake, Databricks, AWS CloudWatch, and the generic REST API) use connector-specific connection details — for example service-account credentials, account/warehouse identifiers, workspace tokens, AWS region/keys, or a base URL with auth headers. The Create New Data Source form shows the fields required for the connector you select.

Step 4: Test Connection

1. Click Test Connection button
2. Wait for validation (typically 2-5 seconds)
3. Verify success message appears

Caution

If connection fails, check:

• Database server is accessible from Beacon application
• Credentials are correct
• Database name exists
• User has at least SELECT permissions

Step 5: Save Data Source

Click Save to create the data source.

Managing Data Sources

View Data Sources

The Data Sources page shows all configured data sources with:

• Data source name and description
• Database type
• Number of queries using this data source
• Last query execution time
• Actions (Edit, Delete, View Queries)

Edit Data Source

1. Click Edit (pencil icon) on the data source row
2. Modify details or connection string
3. Click Test Connection to verify changes
4. Click Save

Delete Data Source

Caution

Deleting a data source will NOT delete associated queries, but queries will be unable to execute.

1. Click Delete (trash icon) on the data source row
2. Confirm deletion in the dialog
3. Data source is archived (soft delete)

Metadata Loading Options

For database-type data sources, Beacon loads schema metadata (tables, columns, relationships) to power features like the ad-hoc query editor with IntelliSense and AI documentation generation. You can control this behavior when creating or editing a data source.

Configuration Options

Option	Description	Default
Metadata Loading Enabled	Enable/disable automatic metadata loading	Enabled
Max Tables	Limit the number of tables loaded (0 = unlimited)	0
Max Columns Per Table	Limit columns per table (0 = unlimited)	0
Table Names Only	Load only table names, skip column details	Off
Include Schemas	Only load metadata from these schemas	All
Exclude Schemas	Skip metadata from these schemas	None

When to Disable Metadata Loading

• Very large databases (1000+ tables) where loading metadata is slow
• Restricted access databases where the user doesn’t have schema read permissions
• CloudWatch or non-database data sources (metadata is not applicable)

When to Use Schema Filters

• Exclude system schemas like information_schema, pg_catalog to reduce noise
• Include only specific schemas to focus on relevant tables
• Multi-schema databases where you only need metadata from certain schemas

Viewing Metadata Status

The data source details page shows the metadata loading status in the Overview section:

• Enabled (green) - Metadata is loaded and available for IntelliSense
• Disabled (red) - Metadata loading is turned off

Example: Large Database with Schema Filtering

When adding a large production database, limit metadata to relevant schemas:

Metadata Loading: Enabled
Max Tables: 200
Include Schemas: public, app
Exclude Schemas: pg_catalog, information_schema, pg_toast

This loads only tables from the public and app schemas, capped at 200 tables.

Schema Relationships

Loaded metadata tells Beacon what tables exist; schema relationships tell it how to join them. Open a data source and go to Schema relationships (/data-sources/{id}/relationships) to review and curate them.

• Foreign key relationships come from the database’s declared constraints and are verified by default.
• Inferred relationships are proposed from column naming with a confidence score. Click Discover to preview proposals without saving, then accept the ones that are correct.
• Manual relationships are declared by hand and are treated as verified.

Only verified relationships are presented to the SQL generator as authoritative join paths; unverified inferences are passed separately and explicitly flagged. The page opens with a schema health panel — table and relationship counts, how many are unverified, connected groups, isolated tables, and detected junction tables — which is the fastest way to see which parts of a schema an AI assistant currently has no way to reach.

See the Knowledge Base guide for how join paths are used during generation, and the REST endpoints for managing them.

Connection Best Practices

Use Read-Only Users

Create dedicated read-only database users for Beacon:

PostgreSQL:

CREATE USER beacon_readonly WITH PASSWORD 'strong-password';
GRANT CONNECT ON DATABASE your_database TO beacon_readonly;
GRANT USAGE ON SCHEMA public TO beacon_readonly;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO beacon_readonly;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO beacon_readonly;

SQL Server:

CREATE LOGIN beacon_readonly WITH PASSWORD = 'strong-password';
USE your_database;
CREATE USER beacon_readonly FOR LOGIN beacon_readonly;
ALTER ROLE db_datareader ADD MEMBER beacon_readonly;

MySQL:

CREATE USER 'beacon_readonly'@'%' IDENTIFIED BY 'strong-password';
GRANT SELECT ON your_database.* TO 'beacon_readonly'@'%';
FLUSH PRIVILEGES;

Enable Connection Pooling

For high-frequency queries, enable pooling:

PostgreSQL:

Host=postgres;Database=db;Username=user;Password=pass;Pooling=true;MinPoolSize=5;MaxPoolSize=20

Benefits:

• Faster query execution (reuse connections)
• Lower database server load
• Better handling of concurrent subscriptions

Set Appropriate Timeouts

For long-running queries, increase timeout:

PostgreSQL:

Host=postgres;Database=db;Username=user;Password=pass;CommandTimeout=300

SQL Server:

Server=sqlserver;Database=db;User Id=user;Password=pass;Connection Timeout=300

Match the timeout with subscription timeout setting.

Examples

Example 1: Production PostgreSQL

Name: Production App Database
Description: Main application database for monitoring
Database Type: PostgreSQL
Connection String: Host=prod-postgres.company.com;Database=appdb;Username=monitor;Password=secret;SSL Mode=Require;Pooling=true;MaxPoolSize=10

Example 2: SQL Server Data Warehouse

Name: Data Warehouse
Description: Analytics database for reporting
Database Type: SQL Server
Connection String: Server=dwh.company.com;Database=analytics;User Id=reporting;Password=secret;TrustServerCertificate=True

Example 3: Multi-Tenant MySQL

Name: Tenant Database (Customer A)
Description: Customer A's isolated database
Database Type: MySQL
Connection String: Server=mysql.company.com;Database=tenant_a;Uid=readonly;Pwd=secret;SslMode=Required

Troubleshooting

Connection Test Fails

Check network connectivity:

# Test database host connectivity
ping your-database-host

Verify database is accessible:

telnet your-database-host 5432  # PostgreSQL
telnet your-database-host 1433  # SQL Server
telnet your-database-host 3306  # MySQL

Common issues:

• Firewall blocking connection
• Database not accepting remote connections
• Wrong hostname or port
• VPN required but not connected

Permission Denied

Ensure database user has necessary grants:

-- PostgreSQL: Check user permissions
SELECT grantee, privilege_type
FROM information_schema.role_table_grants
WHERE grantee = 'your_user';

-- SQL Server: Check user role
SELECT dp.name AS UserName, dp.type_desc, r.name AS RoleName
FROM sys.database_principals dp
LEFT JOIN sys.database_role_members drm ON dp.principal_id = drm.member_principal_id
LEFT JOIN sys.database_principals r ON drm.role_principal_id = r.principal_id
WHERE dp.name = 'your_user';

SSL Certificate Errors

For self-signed certificates:

PostgreSQL:

Host=postgres;Database=db;Username=user;Password=pass;SSL Mode=Require;Trust Server Certificate=true

SQL Server:

Server=sqlserver;Database=db;User Id=user;Password=pass;TrustServerCertificate=True

Related Documentation

• Queries - Create queries using this data source
• Data Migration - Move data between connected sources
• Knowledge Base & Grounding - Join paths, glossary, and the grounding context built from this metadata
• Configuration - Connection string reference