Database connections

A connection is a saved, testable database target that a database source or a Python connector reads through. It is stored as a secret with a readable half: anyone with access to the workspace can see where it points (host, port, database, role, TLS posture, load limits) — nobody can read the password back out, and nobody can repoint the credential without re-entering it.

Create one

Ask the agent, or use Secrets → New connection in the workspace. Either way the flow is the same: the non-secret half is proposed, and you enter the password in a form that opens pre-filled with the destination. What you submit is what gets stored.

core_request_secret_handoff key=ANALYTICS_DB type=postgres
  connection={host: "db.example.com", database: "app", user: "panels_reader", tls_mode: "verify-full"}

The form shows the host, port, database and role the agent proposed. Correct anything that is wrong, paste the CA certificate if your server uses a private one, type the password, and store. Then test it:

core_test_connection key=ANALYTICS_DB

The test walks resolve → dns → tcp → tls → auth → permission and names the first stage that fails — “nothing accepted a TCP connection on port 5432” and “the server’s certificate is not signed by a public CA” are different problems with different fixes, and the report says which one you have.

TLS

Every connection is encrypted unless you say otherwise. With no tls_mode, Panels connects with require (Postgres) / required (MySQL): the connection is encrypted, the server’s certificate is not verified. The other modes:

Mode	What it checks	When
`require` / `required`	encryption only	default; fine when you trust the network path
`verify-ca` / `verify_ca`	the certificate chains to a trusted CA	your server has a certificate from a private CA and you connect by IP
`verify-full` / `verify_identity`	chain and hostname	the recommended mode for anything reachable over the internet
`disable` / `disabled`	nothing	only for a database on a private network you fully control; the UI warns

prefer / allow / preferred are accepted but read as unencrypted: they fall back to plaintext silently if the server declines TLS. The connection list shows encrypted: false for them.

Private CAs

verify-ca and verify-full trust the public CAs out of the box. Managed and self-hosted databases usually sign with their own CA, so paste it into the connection’s CA certificate field (it is public material, stored with the connection and readable back). Panels writes it to a private file inside the execution boundary for the lifetime of the sync and removes it afterwards; a Python connector gets it on get_secret("ANALYTICS_DB")["ca_file"] and already applied on ["url"]. Where to get it:

Amazon RDS / Aurora — download the regional bundle (https://truststore.pki.rds.amazonaws.com/<region>/<region>-bundle.pem) and paste the whole bundle. Connect by the instance endpoint name, not an IP.
Google Cloud SQL — the server CA from the instance’s Connections → Security page (“Download server-ca.pem”). Cloud SQL certificates are issued for the instance name, so use verify-ca unless you connect by that name.
Supabase — the project’s CA from Settings → Database → SSL configuration; connect to the direct host or the pooler in session mode.
Self-hosted Postgres / MySQL — the CA that signed the server certificate (ssl_ca_file in postgresql.conf; ssl_ca in my.cnf).
CloudNativePG — the cluster’s <cluster>-ca Secret, key ca.crt: kubectl get secret <cluster>-ca -o jsonpath='{.data.ca\.crt}' | base64 -d.

Choose what leaves your database

You do not need a view, an index or any schema change to connect. Pick tables and columns on the Panels side; the selection is pushed to your server as the query, so unselected columns never leave it:

{"source_type": "database", "database": {
  "secret_key": "ANALYTICS_DB", "schema": "public", "table": "users",
  "columns": [
    {"name": "id"},
    {"name": "email", "action": "hash"},
    {"name": "ssn",   "action": "exclude"}
  ],
  "watermark": {"column": "updated_at", "type": "timestamp"}
}}

include (default) — the column as-is.
hash — the value is replaced on your server by a SHA-256 of value + a salt private to your workspace, so users.id and orders.user_id still join while the value itself never leaves.
exclude — recorded so the selection documents what stays behind.
A column added to the table later is not picked up until you list it.

core_introspect_connection (or the table picker in the workspace) lists the schemas, tables and columns the connection’s role can see.

What each sync mode can and cannot see

Mode	Sees	Does not see
Full (default)	everything, every time	—
Incremental (`watermark`)	rows whose cursor column advanced since last sync	deletes; updates that do not move the cursor
CDC	everything, including deletes	not offered yet: needs `wal_level=logical` and a replication slot on your server

Incremental is fast only when the cursor column is indexed — adding an index is a change to your database; without one, each sync scans the table. For a table with no single timestamp, give Panels an expression instead of a view:

"computed_watermark": "GREATEST(created_at, updated_at)",
"watermark": {"column": "synced_at", "type": "timestamp"}

To reconcile deletes on an incremental source, schedule an occasional full run, or run the node with Re-sync from scratch.

Read-only role: copy-paste SQL

Zero-change path first. Postgres (RDS, Aurora, Cloud SQL, Supabase, Neon, self-hosted, CloudNativePG all accept this as-is):

CREATE ROLE panels_reader LOGIN PASSWORD '…';
GRANT CONNECT ON DATABASE app TO panels_reader;
GRANT USAGE ON SCHEMA public TO panels_reader;
GRANT SELECT ON public.users, public.orders TO panels_reader;   -- the tables you choose
-- optional, enforced by the database rather than by Panels:
GRANT SELECT (id, created_at, plan) ON public.users TO panels_reader;
-- keep the role harmless:
ALTER ROLE panels_reader SET statement_timeout = '5min';
ALTER ROLE panels_reader CONNECTION LIMIT 8;

MySQL:

CREATE USER 'panels_reader'@'%' IDENTIFIED BY '…';
GRANT SELECT ON app.users TO 'panels_reader'@'%';
GRANT SELECT (id, created_at, plan) ON app.orders TO 'panels_reader'@'%';  -- column-level
ALTER USER 'panels_reader'@'%' WITH MAX_USER_CONNECTIONS 8;

Where the read replica is: RDS/Aurora — the reader endpoint (…cluster-ro-….rds.amazonaws.com); Cloud SQL — a read replica instance’s own IP/name; Supabase — read replicas on the Pro plan, connect to the replica’s host; Neon — a read-only compute endpoint on the branch; self-hosted / CloudNativePG — the -ro service (<cluster>-ro).

Load limits

A sync must not be able to degrade the database it reads. Every connection runs under a load policy, visible on the connection:

Setting	Default	Ceiling
connections a sync may open	4	16
statement timeout	5 min	60 min
lock timeout	5 s	—
idle-in-transaction timeout	60 s	—

Raise or lower them per connection (connection.load on the hand-off, or the form’s Max connections / Statement timeout). A timeout fails the run with a message naming the limit; it never leaves a partial sync unreported. Two things worth knowing:

Point Panels at a read replica where you have one. A sync’s reads pin the vacuum horizon on a primary and can queue a migration behind them; the connection test tells you which kind of server it found.
Index your watermark column. Incremental syncs filter on it, and the change probe runs SELECT MAX(watermark) — cheap with an index, a table scan without.

Long syncs

A run is capped at a few minutes, so a large initial sync is a chain of runs: each resumes from the cursor the previous one checkpointed, and Panels dispatches the next one itself until the sync is done. Run history shows the chain as one sync. If a sync must start over, run the node with Re-sync from scratch (run_node reset_sync_state=true): every cursor is cleared and the next run is a full sync. For a Python connector, the same thing is one call:

def fetch(ctx):
    for page in pages(since=ctx.state.get("cursor")):
        ctx.upsert(page.rows, dedup_keys=["id"])
        ctx.checkpoint({"cursor": page.next_cursor})
        if ctx.budget_remaining() < 30:
            ctx.request_continuation("time budget")

Private databases: through an SSH bastion

If the database is not reachable from the internet, expose only an SSH bastion and let Panels tunnel through it. Choose Through an SSH bastion in the connection form (or route: "ssh_tunnel" with ssh: {host, user, port?} on the hand-off). Panels mints one ed25519 keypair per connection; the private key never leaves Panels, and the public key is shown once the connection is stored (and on the connection afterwards) for you to install on the bastion.

Set up the bastion

Create a dedicated user, e.g. panels, with no shell access needed. On Debian and Ubuntu, useradd leaves the new account locked, and OpenSSH refuses a locked account even for key-only logins (the bastion logs User panels not allowed because account is locked). Give it a password field that is unusable but not locked:
sudo useradd --create-home --shell /usr/sbin/nologin panels sudo usermod -p '*' panels
Install the public key in that user’s ~/.ssh/authorized_keys with restrictions, so the key can only ever forward to this one database:

permitopen="db.internal:5432",no-pty,no-agent-forwarding,no-X11-forwarding,no-user-rc,command="/bin/false" ssh-ed25519 AAAA… panels-postgres-db.internal

Allow Panels’ egress addresses on the bastion’s SSH port; the database port never opens to the internet.
Run Test connection. The test walks bastion → SSH auth → host key → forward before the usual database stages, and on the first success it pins the bastion’s host key. Every sync verifies the bastion against that pin; a changed key fails closed with a message naming both fingerprints. If you rebuild the bastion, re-create the connection (which also mints a new keypair — that is how a key is rotated).

TLS to the database still applies end to end inside the tunnel, and for Postgres verify-full checks the database’s real name, not localhost. MySQL’s driver verifies the chain (verify_ca) but not the name through a tunnel. Python connectors get the same behaviour with no code change: the runner opens the tunnel before your code runs, and get_secret("ANALYTICS_DB")["url"] already points through it.

Private databases: through a Cloudflare Tunnel

If you already run Cloudflare Tunnel, publish the database as an Access-protected TCP application and Panels reaches it through Cloudflare’s edge. Nothing listens on the internet, and the database needs no firewall change. Choose Through a Cloudflare Tunnel (Access) in the connection form (or route: "cloudflare_access" with cloudflare: {hostname} on the hand-off).

Set up the tunnel

Add a public hostname to the tunnel whose service is the database over TCP, for example db.example.com → tcp://db.internal:5432.
In Zero Trust, create an Access application for that hostname, and a service token for Panels only. Give it an expiry.
Add a policy to the application with action Service Auth that includes that service token. For defence in depth, add an IP ranges rule with the addresses Panels connects from.
In the connection form, enter the tunnel hostname, then the token’s client ID and client secret beside the database password. Keep Host as the database’s own name (db.internal): the tunnel decides where the hostname goes, and TLS verifies the real name end to end inside it.
Run Test connection. The test checks Cloudflare Access before the usual database stages: a refused token, a missing Service Auth policy and a disconnected tunnel each fail with their own message.

The service token is stored like a password: it is never shown again, and rotating it means re-creating the connection. Python connectors cannot use this route yet; use a DuckDB database source.

Private databases: over your tailnet

If the database is on a Tailscale (or Headscale) tailnet, Panels joins as an ephemeral, tagged node for the length of each sync and test, then leaves. Choose Over our tailnet in the connection form (or route: "tailscale" with tailscale: {tags, client_id?} on the hand-off). Host is the database’s tailnet name or address, or an address behind an approved subnet router.

In the tailnet policy, make a tag for Panels (e.g. tag:panels) and grant it only the database’s port: {"src": ["tag:panels"], "dst": ["db:5432"]}.
Preferred — federated identity, nothing stored. In Tailscale’s admin console, add a federated identity with a custom issuer:
- Issuer: https://app.panels-analytics.com/v1/workload-identity
- Subject: the connection’s subject, shown on the connection after you save it (tenant:…:conn:…). Pin it exactly.
- Tags: tag:panels.
Enter its client ID in the form.
Otherwise (Headscale, or no federation on your plan), create a reusable, ephemeral, tagged auth key for tag:panels and enter it in the form. It is stored like a password.

Private databases: through AWS Systems Manager

For a database in a private VPC (RDS, Aurora, or self-hosted), Panels assumes a role in your account with its own short-lived identity and forwards through an SSM-managed instance. Nothing is stored, and nothing opens to the internet. Choose Through AWS Systems Manager (or route: "aws_ssm" with aws_ssm: {role_arn, region, target}).

Add an IAM OIDC identity provider for https://app.panels-analytics.com/v1/workload-identity, audience sts.amazonaws.com.
Create a role that trusts it for exactly this connection:

{
  "Effect": "Allow",
  "Principal": { "Federated": "arn:aws:iam::<account>:oidc-provider/app.panels-analytics.com/v1/workload-identity" },
  "Action": "sts:AssumeRoleWithWebIdentity",
  "Condition": { "StringEquals": {
    "app.panels-analytics.com/v1/workload-identity:sub": "tenant:…:conn:…",
    "app.panels-analytics.com/v1/workload-identity:aud": "sts.amazonaws.com"
  } }
}

Allow the role only ssm:StartSession on the one instance and on the AWS-StartPortForwardingSessionToRemoteHost document, and ssm:TerminateSession on its own sessions.
Enter the role ARN, region and instance ID. After saving, the connection shows the exact subject to put in the trust policy; then run Test connection (AWS role → SSM session before the database stages).

Each database connection is its own SSM session. SSM sessions are slower than a direct link (about 1 MB/s each), so first loads of large tables take a while; incremental syncs are unaffected.

Where Panels connects from

Every database connection — database sources, Python connectors that use a connection, and SSH bastion routes — leaves Panels from one of two fixed addresses, whichever machine runs the sync:

Address
`91.98.106.218`
`91.98.103.132`

Allowlist both on your firewall or security group rather than opening the port to the internet; a sync may use either. The connection test’s tcp stage is the quickest way to confirm the allowlist is right. The same list is published for automation at https://panels-analytics.com/egress-ips.json. Changes. We give 30 days’ notice, by email to workspace owners and in the JSON file, before removing or replacing an address. Adding an address needs no action from you until the notice says so. Shared addresses. These addresses are shared by all Panels workspaces, so an allowlist entry tells your database the connection came from Panels, not from your workspace in particular. Keep authenticating with a dedicated read-only role and TLS as described above.

How It Works

Notebooks

Visualization

Data

Automation

Reliability

AI

Create one

TLS

Private CAs

Choose what leaves your database

What each sync mode can and cannot see

Read-only role: copy-paste SQL

Load limits

Long syncs

Private databases: through an SSH bastion

Set up the bastion

Private databases: through a Cloudflare Tunnel

Set up the tunnel

Private databases: over your tailnet

Private databases: through AWS Systems Manager

Where Panels connects from

​Create one

​TLS

​Private CAs

​Choose what leaves your database

​What each sync mode can and cannot see

​Read-only role: copy-paste SQL

​Load limits

​Long syncs

​Private databases: through an SSH bastion

​Set up the bastion

​Private databases: through a Cloudflare Tunnel

​Set up the tunnel

​Private databases: over your tailnet

​Private databases: through AWS Systems Manager

​Where Panels connects from

Create one

TLS

Private CAs

Choose what leaves your database

What each sync mode can and cannot see

Read-only role: copy-paste SQL

Load limits

Long syncs

Private databases: through an SSH bastion

Set up the bastion

Private databases: through a Cloudflare Tunnel

Set up the tunnel

Private databases: over your tailnet

Private databases: through AWS Systems Manager

Where Panels connects from