Every table in a Panels project that isn’t produced by a SQL or Python transformation enters through a source node — a node that pulls or receives data from outside the computation graph. Rather than shipping a per-vendor connector catalog, Panels exposes a small set of source kinds that compose to cover any system. This page explains that design, how pull and push sources differ, how credentials stay out of the graph, and how ingested data ends up as queryable tables.

​
How it works

​
No per-vendor connector catalog

Building a dedicated connector for every vendor — a Snowflake connector, a BigQuery connector, a Salesforce connector, each with its own config schema, auth flow, and version to maintain — doesn’t scale. Panels avoids that treadmill by exposing ingestion primitives instead of connectors, and letting the AI agent compose them for whatever system you’re connecting to. Two primitives cover pulling data from an external system:
  • database — for network databases that support a direct connection: Postgres and MySQL. Panels connects read-only and pushes filters and projections down to the source database, so performance comes from the connection itself rather than from tuning.
  • python — for everything else: Snowflake, BigQuery, Redshift, REST APIs, and SaaS tools. A python source runs a sandboxed fetch() function that calls the vendor’s SDK or HTTP API and streams rows out. When you ask the AI agent to connect to a system that isn’t Postgres or MySQL, it writes this function for you.
The routing rule: Postgres or MySQL uses a database source; everything else uses a python source. There’s no “Snowflake connector” or “BigQuery connector” to configure — asking for one produces a python source under the hood. Adding support for a new vendor is a matter of the AI writing a new fetch() function, not platform engineering work.
If you ask the AI agent to connect to a SaaS tool or a cloud data warehouse, it writes a python source for you. You don’t need to know in advance which primitive a given vendor maps to.

​
Source kinds

Every source node has exactly one kind, which determines how it reaches its data:

​
Pull vs push

Source kinds split into two models based on who initiates a sync. Pull sources (database, python, file, object_storage) run on a schedule you configure on the node: manual (run on demand), cron (a recurring interval), or on-change (triggered when something upstream changes). Panels initiates the connection and fetches data at that trigger. Push sources (webhook, email, upload) are event-driven. There’s no schedule to configure — the node runs when the external system sends data:
  • A webhook source gets a server-minted URL and bearer token. Panels writes every incoming request to a durable inbox before processing it, and deduplicates deliveries so retried or duplicate requests don’t double-ingest.
  • An email source gets a minted email address. You configure a sender allowlist to control who can deliver data, and Panels can parse the message body, its attachments, or both depending on the parse mode you choose.
  • An upload source ingests files you drop into the project directly, including multiple files in one upload.
Push sources are event streams: the sender controls the shape of each payload, and that shape can change between events. Panels stores every field from a webhook or email source as text so a sync never fails because a field’s type changed. Learn how to convert that into typed columns in Event sources & typing.

​
Credentials and secrets

Neither primitive stores credentials in the computation graph. A database source references a stored secret by key, which the worker resolves to a connection string only at run time. A python source references named secrets that are resolved into environment variables inside the sandboxed run. Credentials themselves live encrypted at the team level, not inside the graph definition. The graph validator enforces this: it rejects any source configuration that contains an inline credential (a raw password, API key, or connection string) instead of a secret reference.
Panels rejects a graph write if a source node’s configuration carries a credential value directly instead of a secret_key reference. Store the credential as a secret first, then reference it by key.

​
Key behaviors

​
Materialization into DuckLake

Every source, regardless of kind, lands its output as a table in DuckLake, the storage layer that backs every Panels project. That table is the cache: downstream SQL and Python nodes read it directly rather than re-fetching from the original system on every run. Because DuckLake tables participate in reactive execution, a new sync that changes a source’s output propagates downstream automatically — dependent nodes re-run against the fresh data without you triggering them manually.

​
Data going the other way: writeback

Source nodes bring data into Panels. Moving data back out — writing query results to an external database, calling an API, or delivering a file to an external system — is a separate node type called a writeback node, with its own delivery modes, retry behavior, and audit trail.
A dedicated writeback guide is planned but not yet published in these docs.

​
Implications for your workflow

Because there’s no connector catalog, you don’t pick a vendor from a list — you describe what you want to connect to, and the AI agent chooses database or python and writes the configuration or fetch code. For Postgres and MySQL, expect that to happen with no code to review. For anything else, expect a python source whose fetch() function you can read and edit like any other node. When you configure incremental sync, choose a watermark column that’s monotonically increasing (a timestamp or auto-incrementing ID) — Panels uses it to fetch only rows added or changed since the last successful sync. If a pull source supports a change probe, leave full syncs on their normal schedule; the probe skips the expensive fetch on runs where nothing changed, so tightening the schedule doesn’t cost you sync time. For push sources, treat the raw table as a landing zone rather than a final table. Add a downstream SQL node that casts fields to their real types before building dashboards on top, per Event sources & typing.

​
FAQ

Not as a separate product. Both are reached through a python source — the AI agent writes a fetch() function that calls the vendor’s SDK. Functionally this behaves like a dedicated connector, but there’s no separate catalog entry or config schema to maintain per vendor.
Panels’ graph validator rejects inline credentials. Store the credential as a secret first, then reference it by key from the source configuration. This keeps credentials out of the graph definition, project history, and any exported or shared copy of the graph.
Webhook sources deduplicate incoming requests before processing, so a retried delivery from the sending system doesn’t create duplicate rows. Sources with declared dedup keys use upsert semantics, so re-processing the same record is safe even if deduplication at the transport layer doesn’t catch it.
Yes, for sources that support a change probe. Panels checks a cheap signal (an ETag, last-modified timestamp, or row count) before running the full fetch, and skips the sync when the signal matches the last run.
  • DuckLake — where every source’s data lands, and how versioning and schema evolution work
  • Execution model — how a fresh sync propagates to downstream nodes
  • Event sources & typing — converting push-source event data into typed columns