Extract data from emailed receipts

This guide stitches together four pieces of Panels — the email source, blob input files, Claude vision, and table output — into a working pipeline that turns photos (or PDFs) of receipts into a queryable table. Forward (or email) receipts to a project address; a Python node reads each attachment, calls Claude to extract structured fields, and writes one row per receipt — with versioning columns that make the table reproducible and safely re-runnable. The full worked example lives at examples/receipt-extraction/ (extract_receipts.py, a stubbed-vision test, and a manual end-to-end script).

The blob attachments this guide reads with input_files() work the same whether they arrive by email or browser upload — the same storage model, the same content_hash dedup key, and the same receipts_inbox index table.

How it flows

 email w/ image or PDF attachments
        │
        ▼
 ┌──────────────┐   registers each attachment as a       ┌───────────────────────┐
 │ Email source │   blob artifact on its output  ──────▶ │ Python transform node │
 │    node      │   (receipts_inbox metadata table,      │  extract_receipts.py  │
 └──────────────┘    incl. a content_hash per blob)       └───────────┬───────────┘
                                                                      │
                       input_files()  ◀── worker stages               │  output_table(rows)
                       reads staged bytes + content_hash              ▼
                       (no network needed)                     ┌────────────────┐
                                                               │ DuckLake table │
                       Claude (api.anthropic.com)              │ one row/receipt│
                       extract → flatten → versioned rows      │ append + upsert│
                                                               └────────────────┘

Email source node — receives the email. Attachments (PNG/JPEG/PDF/etc.) are registered as blob artifacts on the node’s output, and a receipts_inbox metadata table (one row per attachment, including a content_hash — the sha256 of the bytes) is emitted for preview and lineage. The raw bytes are uploaded format-agnostically by the SMTP ingest boundary, so non-tabular formats are not garbled. Blobs are immutable and content-addressed — this is the bronze layer everything downstream can be re-derived from.
The worker stages those blobs to local files before the downstream Python node runs, and exposes them through the input_files() SDK helper — each InputFile carries the bytes, filename, content type, and content_hash. Reading them needs no network access and no read credentials.
Python transform node — input_files() enumerates the staged receipts; for each (up to 4 concurrently), the node base64-encodes the bytes and calls the Claude Messages API with a structured-extraction prompt — an image content block for images, a document content block for PDFs.
output_table(rows) — the flattened rows (one per receipt) are materialized as a DuckLake table you can query or visualize. With the node configured for append + dedup (below), the table accumulates across runs as a silver layer.

1. Set up the email source node

Add an email source node to your project and copy its ingest address (it looks like <hookid>@ingest.<your-domain>). Anything emailed to that address lands on the node. Send a test email with one or more receipts attached; the node’s receipts_inbox table should show one row per attachment, each with its content_hash.

2. Add the `ANTHROPIC_API_KEY` secret

The Python node reads the key with get_secret("ANTHROPIC_API_KEY"):

Open the project’s Secrets panel.
Add a secret named ANTHROPIC_API_KEY with your Anthropic API key as the value.
Attach the secret to the Python node (add it to the node’s secret refs).

The worker-python network policy already allows outbound HTTPS to api.anthropic.com, so the vision call works once the secret is set. The key is injected into the sandbox config and is never written to disk or logged.

3. Add the Python transform node

Create a Python transformation node downstream of the email node and paste the body of extract_receipts.py, ending the script with an explicit run() call. Then set two node params (this is what makes the table accumulate — see Accumulating across runs):

output_mode = "append"
dedup_keys = ["content_hash", "prompt_version"]

The node, in outline:

import base64, hashlib, json, urllib.request
from concurrent.futures import ThreadPoolExecutor
from _panels import input_files, get_secret, output_table

MODEL = "claude-sonnet-4-6"   # fast/cheap vision tier — plenty for receipts
PROMPT_VERSION = "v1"         # bump when EXTRACTION_PROMPT changes
MAX_RECEIPTS_PER_RUN = 12     # bound calls to the run timeout
MAX_WORKERS = 4               # bounded parallelism over receipts

def fetch_one(f, api_key):
    data = f.read_bytes()
    # content_hash comes from the staging manifest; hash locally as a
    # fallback so pre-checksum artifacts still work.
    chash = f.content_hash or hashlib.sha256(data).hexdigest()
    media_type = f.content_type or "image/png"
    block = (
        {"type": "document", "source": {"type": "base64",
         "media_type": "application/pdf", "data": b64(data)}}
        if media_type == "application/pdf" else
        {"type": "image", "source": {"type": "base64",
         "media_type": media_type, "data": b64(data)}}
    )
    try:
        response = post_messages(MODEL, [block, prompt_block], api_key)
        return (f.filename, chash, response, "")
    except Exception as exc:
        return (f.filename, chash, None, f"vision call failed: {exc}")

receipts = input_files()[:MAX_RECEIPTS_PER_RUN]
with ThreadPoolExecutor(max_workers=MAX_WORKERS) as pool:
    futures = [pool.submit(fetch_one, f, get_secret("ANTHROPIC_API_KEY"))
               for f in receipts]
    results = [fut.result() for fut in futures]   # input order preserved

output_table(flatten_rows(results))   # ok rows + per-receipt error rows

The example file is more defensive than this sketch (string-number coercion, code-fence tolerance, an empty-table branch, error-message truncation, a loud warning when receipts are skipped), and splits the network call from the data-shaping so the shaping can be unit-tested without a network.

What each receipt extracts to

One output row per receipt — successful or not — with line_items serialized to a JSON string column so the row stays flat and DuckLake-friendly:

column	example	notes
`source_file`	`receipt_1_the.png`	attachment filename
`content_hash`	`9f86d08…` (64 hex chars)	sha256 of the receipt bytes; dedup key
`vendor`	`THE DAILY GRIND`
`date`	`2026-06-09`
`total`	`14.76`
`currency`	`EUR`
`payment_method`	`VISA`
`line_item_count`	`4`
`line_items`	`[{"name":"Flat White","qty":1,…}]`	JSON string column
`model`	`claude-sonnet-4-6`	model id used for this row
`prompt_version`	`v1`	dedup key; bump on prompt changes
`extracted_at`	`2026-06-12T08:30:00Z`	UTC ISO timestamp of the run
`status`	`ok` / `error`
`error`	`""` / `vision call failed: …`	truncated failure message

Downstream, explode line_items with DuckDB’s json_extract / UNNEST, or keep it as-is for a per-receipt summary. Filter status = 'ok' for analysis queries; keep an eye on status = 'error' rows for receipts that need a retry or a better photo.

Per-receipt error isolation

One bad receipt does not abort the batch. Each receipt’s vision call and JSON parse are isolated: on failure the node emits a row with status = 'error', the truncated message in error, and the receipt’s identity columns (source_file, content_hash) — and continues with the rest. Because the error row carries the same dedup keys, a later re-run that succeeds for that receipt upserts over the error row.

Accumulating across runs (append + dedup)

The accumulation behavior is node configuration, not SDK code. Set on the Python node:

output_mode = "append"
dedup_keys = ["content_hash", "prompt_version"]

Semantics:

Re-runs upsert. A receipt already extracted with the same bytes (content_hash) and the same prompt (prompt_version) replaces its previous row instead of duplicating it — re-running the node, re-sending an email, or retrying after errors is always safe.
Bumping PROMPT_VERSION re-extracts everything under the new version alongside the old rows. Compare v1 vs v2 extractions side by side, then filter on prompt_version downstream once you trust the new prompt.
Silver is always re-derivable. The email node’s bronze blob artifacts are immutable and content-addressed, so you can clear the extracted table and re-run from scratch to get identical rows back (modulo extracted_at and model non-determinism).

Why the model id and message shapes are exact

Model: claude-sonnet-4-6. Receipt extraction is simple, well-specified structured vision work — it doesn’t need Opus-tier reasoning, and the Sonnet tier is the current fast/cheap vision-capable choice ( $3/$ 15 per MTok vs $5/$ 25 for claude-opus-4-8) with both image and PDF support. Use the bare id — do not append a date suffix. The model id is recorded in each row’s model column, so a later model change is visible in the data.
Image message shape: a user turn whose content is an image block with a base64 source (media_type + data), followed by the text prompt.
PDF message shape: identical, except the block is {"type": "document", "source": {"type": "base64", "media_type": "application/pdf", "data": …}}. PDFs emailed as receipts are handled automatically; other content types produce a status='error' row.
No sampling params: the example sends no temperature / top_p / thinking config — the minimal request shape works across current models.

Staying inside the run timeout

A Python run has a wall-clock timeout. Each receipt is one vision call of a few seconds; the example runs up to MAX_WORKERS (4) calls concurrently and caps the batch at MAX_RECEIPTS_PER_RUN (default 12). The worker also caps total staged input bytes at ~25 MB per run. If more receipts are staged than the cap, the overflow is skipped this run and the node prints a loud warning naming how many were skipped and why. Send large batches across several emails / runs — with append + dedup configured, the rows accumulate and nothing is double-counted.

Testing it

Without a network or a stack — extract_receipts_test.py feeds the example’s parse/flatten logic canned Anthropic responses built from a ground-truth fixture and asserts the output rows match — including the versioning columns, the PDF document block shape, and the error-isolation path (one bad response among good ones yields one error row and N−1 ok rows). Run it with python3 -m pytest examples/receipt-extraction/extract_receipts_test.py -q.
Live, end-to-end — manual_qa_e2e.py emails the demo receipts to your email node and asserts the extracted vendor/total per receipt against ground truth (querying status = 'ok' rows). It needs a running stack and a real API key, so it is gated behind PANELS_RECEIPT_E2E=1 and skips otherwise.

How It Works

Notebooks

Visualization

Data

Automation

Reliability

AI

Extract data from emailed receipts

How it flows

1. Set up the email source node

2. Add the `ANTHROPIC_API_KEY` secret

3. Add the Python transform node

What each receipt extracts to

Per-receipt error isolation

Accumulating across runs (append + dedup)

Why the model id and message shapes are exact

Staying inside the run timeout

Testing it

​How it flows

​1. Set up the email source node

​2. Add the ANTHROPIC_API_KEY secret

​3. Add the Python transform node

​What each receipt extracts to

​Per-receipt error isolation

​Accumulating across runs (append + dedup)

​Why the model id and message shapes are exact

​Staying inside the run timeout

​Testing it

How it flows

1. Set up the email source node

2. Add the `ANTHROPIC_API_KEY` secret

3. Add the Python transform node

What each receipt extracts to

Per-receipt error isolation

Accumulating across runs (append + dedup)

Why the model id and message shapes are exact

Staying inside the run timeout

Testing it