---
title: "Bucket ingestion"
description: "Point Edge Assist at an Edge Storage bucket: supported formats, the ETag diff cycle, the 10,000-document cap and private access with API keys and tokens."
url: "https://edge.network/docs/assist/bucket-ingestion"
section: "Assist"
---

# Bucket ingestion

The knowledge-base tier: point Assist at an Edge Storage bucket and drop documents in. No index file, no build step: the bucket is the source of truth, and Assist keeps itself in sync with it. Available on pay-as-you-go.

## How it works

1. Create a bucket in **Storage** and upload your documents.
2. In **Assist → Ingestion**, switch the content source to **Storage bucket**, pick the bucket (and an optional key prefix like `docs/`) and save.
3. Assist lists the bucket hourly and diffs it by ETag: new and changed objects are parsed and indexed, deleted objects drop out of the index. Only what changed is re-processed.

There is no push API by design. Uploading, replacing, or deleting objects in the bucket *is* the ingestion interface. Any tool that can write to Edge Storage (the S3-compatible API, the Console, a CI job) can maintain the knowledge base.

> [!NOTE]
> Any content change clears the site's [answer cache](/docs/assist/caching), exactly as with index-file ingestion, and "Re-ingest now" in Control forces a diff immediately.

## Supported formats

| Extension | Notes |
|---|---|
| `.md`, `.markdown` | Front matter is stripped; the first heading becomes the document title; code fences are skipped. |
| `.txt` | Ingested as-is. The filename becomes the title. |
| `.html`, `.htm` | Tags, scripts, and styles are stripped; the `<title>` becomes the document title. |
| `.pdf` | Text extraction only. Scanned or image-only PDFs have no extractable text and are recorded as parse errors. |

Everything else in the bucket is skipped. Documents are chunked into ~1,400-character passages on paragraph and sentence boundaries (the same retrieval unit as the index-file tier) and cited in answers by document name. A document that fails to parse never blocks the rest of the bucket; it's listed with its error on the Ingestion tab and retried automatically when its content changes (or on a forced re-ingest).

## Limits

| Limit | Value |
|---|---|
| Documents per site | 10,000 |
| Size per document | 20 MB |
| Chunks per site | 50,000 |

Knowledge bases beyond 10,000 documents are handled by [Expert Services](/solutions/expert-services), with custom retrieval, access control per prefix, and dedicated capacity.

## Private access

Bucket-sourced sites are **private by default**, because an internal knowledge base shouldn't be one lifted sitekey away from public. A private site rejects anonymous search, ask, and config calls; access comes from per-site **API keys**, managed in **Assist → Settings → Access**. Keys are shown once at creation and stored hashed. (You can flip a site back to public there too. Public sites are protected by the Origin check instead.)

### The widget on a private site

Browsers never see the API key. Your server exchanges it for a short-lived visitor token (15-minute expiry, site-scoped) and renders the token into the embed:

```bash
# Server-side: exchange your API key for a short-lived visitor token
curl -X POST https://assist.edge.network/api/token \
  -H "Authorization: Bearer eak_your_site_api_key" \
  -H "Content-Type: application/json" \
  -d '{"sitekey": "ea_your_sitekey"}'

# → { "token": "eat_…", "expiresIn": 900 }
```

```html
<!-- Render the token into the embed: the API key never reaches the browser -->
<script src="https://assist.edge.network/widget.js"
  data-sitekey="ea_your_sitekey"
  data-token="eat_…" defer></script>
```

Mint the token in the request that serves the page (behind your own login), and only authenticated users can ever reach the assistant. The hosted view works the same way: link to `/w/<sitekey>?token=eat_…`.

### Headless use

Server-side callers can skip tokens entirely and present the API key directly:

```bash
# Headless: the API key works directly on search and ask
curl "https://assist.edge.network/api/search?sitekey=ea_your_sitekey&q=refund+policy" \
  -H "Authorization: Bearer eak_your_site_api_key"
```

## Next steps

- [Content ingestion](/docs/assist/ingestion) — The index-file tier for website content
- [API reference](/docs/assist/api) — Search, ask and token endpoints
- [Edge Storage](/docs/storage) — Buckets, uploads and the S3-compatible API