Source Manifest Reference

A source declares itself in TypeScript. You write one defineSource({ … }) call in extension.ts holding both what the source is and what it does, and manifest.json is generated from it.

extension.ts

import { defineSource } from '@ontrove/extend/source';

export default defineSource({
  id: 'hex-blog',
  name: 'Hex Blog',
  description: 'Data science and analytics engineering posts',
  icon: '⬡',
  version: '0.1.0',
  author: 'Hollyburn Analytics Inc.',
  kind: 'scheduled-sync',
  transport: 'scrape',
  cursor: 'idSet',
  ingest: 'append',
  runsIn: 'cloud',
  schedule: 'weekly',
  status: 'implemented',
  needsBrowser: false,
  egress: ['hex.tech'],
  async sync(ctx) {
    // …fetch the listing and return { documents, cursor }
  },
});

Why it is generated

A hand-written manifest is a declaration nothing compiles. Every one of ours had drifted — manifests carried an sdk field naming versions of a package that had long since moved past them, because no validator, no backend and no client ever read it. Declaring the same facts in TypeScript makes the compiler the thing that notices.

Two consequences worth knowing before you start.

Validation is eager. defineSource validates the manifest half at definition time, so an unrecognised cadence, a runsIn your transport cannot satisfy, or a credential smuggled into config throws when the module is imported. You meet the error at your desk and again at deploy — not on the first scheduled run, unattended, weeks later.

manifest.json is still committed, and you never edit it. The readers cannot execute your source: Trove’s catalog build and the Mac app both read the file off disk, and neither runs TypeScript to do it. So the file exists, carries "generated": true, and is rewritten from your defineSource call on every build. Editing it by hand loses the change on the next one.

trove source init prints the old filename

The scaffold writes extension.ts and a generated manifest.json, but its closing hint still says to edit index.ts. Edit extension.ts. index.ts is the previous convention and is still accepted as a fallback.

Identity

Field	Type	Required	Notes
id	string	Yes	Pattern ^[a-z0-9-]+$. Stable source-type id, unique within a catalog.
name	string	Yes	Display name. Short and descriptive — Hacker News Upvotes, not HN Upvoted Stories Indexer v2.
description	string	Yes	One line, in the directory listing. Say what gets indexed, not how.
icon	string	Yes	A single emoji, or an HTTPS URL to a square icon.
version	string	Yes	Semver. Bump it on each meaningful change.
author	string	Yes	Attribution in the directory.

Renaming id is a breaking change

id is the source’s namespace in storage. Changing it after people have added the source makes Trove treat it as a different source — existing instances and their synced documents do not follow the rename. Choose it before you publish.

The four type-system fields

These describe the source’s collection contract: which entrypoint the harness calls, how the source reaches its data, how it resumes, and what ingest does with what it returns. All four are required, each is checked against its vocabulary, and a source that has code is additionally held to the subset the runtimes build today.

kind

scheduled-sync | on-demand-fetch | on-demand-query

Which entrypoint the harness invokes. scheduled-sync is the batch contract this SDK is built around — a sync(ctx) that returns documents to store. The on-demand kinds describe adapters answering a single question live rather than filling a library.

Built today: scheduled-sync. A source with a sync must declare it.

transport

feed | scrape | api | browser | local

How the source reaches its data. This is what decides whether the source can run anywhere but the user’s own machine — see runsIn below.

cursor

date | idSet | none | highWaterId | opaqueToken | snapshot | mtime | rowid

How the source remembers where it stopped. See Cursors and Feeds for what each strategy means in a sync.

Built today: date, idSet, none. A source deployed as its own package may also use highWaterId, because a deployed source’s cursor is handed back byte-for-byte, while a source compiled into Trove’s own runtime has its cursor parsed on the way in — and a shape the parser does not recognise reads as “no cursor”, which means starting from the beginning on every run, forever, with no error anywhere. That is why the two lists differ rather than the difference being a footnote.

ingest

append | upsert

What ingest does with the documents a run returns. append adds what is new and leaves what is stored alone; upsert lets a later run replace an earlier document carrying the same id, which is what a source whose upstream edits items in place needs.

This is not a content type. A document’s kind is its own contentType field; ingest says only whether a re-run may overwrite.

Built today: append.

Where and when it runs

runsIn

cloud | mac — required.

The default executor, and the eligibility bound. A cloud source can be flipped to run on a particular user’s Mac, because their machine can run anything. The reverse is not possible: a source that needs a browser or a local file cannot be hoisted into a runtime that has neither.

Declaring runsIn: 'cloud' is checked, not taken on trust. It requires:

• transport is feed, api, or scrape — the transports whose sync is a pure HTTP pull. A browser source drives a real browser and a local source reads the user’s disk; neither exists in a hosted runtime.
• needsBrowser is false.

schedule

One of: every 30 minutes, every 1 hour, every 2 hours, every 4 hours, every 6 hours, every 12 hours, daily, weekly, monthly, yearly, on demand.

A closed vocabulary rather than free text, because an unrecognised cadence used to be treated as daily — so a source asking for hourly ran twenty-four times less often than its author believed, and nothing said so.

needsBrowser

boolean — required.

Whether the source needs a real browser session. true pins the source to the Mac app; see runsIn above.

status

string — required.

implemented for a source with code. A source still being designed may be declared without a sync, and may name vocabulary values that are not built yet, to record where its shape is headed. Only a source with code is held to the built subsets above.

What it may reach

egress

string[] — required.

The hosts this source may reach. Host-exact and enforced at run time: a host absent from this list is unreachable, not merely undeclared. No scheme, port, path, or wildcard — a wildcard the SDK accepted and the platform did not would deploy and then fail to reach what it declared.

egress: ['hex.tech', 'cdn.hex.tech'],

An entry may instead be the sentinel config:<field>, naming a config field whose value supplies the host — the shape a generic feed reader needs, where the user brings the URL.

config: { feedUrl: { label: 'Feed URL', type: 'url' } },
egress: ['config:feedUrl'],

The sentinel is rejected for a source deployed as its own package, whose allowlist is host-exact and fixed at deploy time.

Field	Type	Notes
egressNote	string	Why the list looks the way it does, when that needs saying.
egressNotFetched	string[]	Hosts that appear in content but are never fetched. Hostnames only — the config: sentinel names hosts the adapter does fetch.

historyReach

{ kind: 'full' | 'window' | 'recent-only', note: string } — optional.

How much of the upstream is reachable at all. This is a property of the upstream, not of your adapter: an RSS feed carrying the last twenty posts cannot be made to yield the twenty-first by syncing harder. Declaring it lets the product explain why a backfill stopped where it did, instead of presenting a bounded archive as a complete one.

historyReach: { kind: 'window', note: 'The feed carries the most recent 20 posts.' },

What the user fills in

config

Record<string, ManifestConfigField> — optional.

The preference fields shown during setup. Each key becomes a property on ctx.config; each value describes the input. Field descriptors take label, type, placeholder, pattern, hint, default, and an optional directory descriptor that gives the field a searchable picker instead of free text. Field-by-field types are in the generated API reference.

config: {
  username: { label: 'HN Username', type: 'text', placeholder: 'pg' },
  feeds: { label: 'Feed URLs', type: 'url[]' },
},

Preferences only — never credentials

config holds user preferences: feed URLs, usernames, section lists, filters. It is stored in the cloud, and Trove rejects writes whose config contains credential-shaped keys — anything containing session, cookie, token, api_key, password, secret, auth, credential, bearer, or a key or private key, matched case- and separator-insensitively so apiKey, api_key, and access_token all match. The SDK lints your manifest against the same list, so a config it accepts is one the platform accepts.

Credentials are named in secrets and read through ctx.secret(). They never travel in config.

fanOut

string — optional.

The name of the config field this source fans out over: one run covers every value in it, one feed per entry. The named field must be a list — url[] or text[] — because a scalar cannot fan out, and naming one is rejected rather than silently producing a single feed.

config: { feeds: { label: 'Feed URLs', type: 'url[]' } },
fanOut: 'feeds',

secrets

string[] — optional.

The credential names this source reads through ctx.secret(name). Values are held outside the source and outside config, and resolved at sync time — see ExtensionContext.secret in the generated API reference.

How its documents are handled

Field	Type	Notes
formatting	reformat | verbatim	Whether Trove restructures the body into clean Markdown — headings, paragraph breaks, lists — while preserving the words, or stores it exactly as received. Optional, and defaults to verbatim, so a new or third-party source never has its data altered unless its author opts in.
available	boolean	Whether the source is offered in the marketplace.
generated	boolean	Written into manifest.json to mark it an artifact. You never set this.

The minimum

Every required field, and nothing else:

extension.ts

import { defineSource } from '@ontrove/extend/source';

export default defineSource({
  id: 'my-blog',
  name: 'My Blog',
  description: 'Posts from my blog.',
  icon: '📄',
  version: '1.0.0',
  author: 'you',
  kind: 'scheduled-sync',
  transport: 'feed',
  cursor: 'date',
  ingest: 'append',
  runsIn: 'cloud',
  schedule: 'daily',
  status: 'implemented',
  needsBrowser: false,
  config: { feedUrl: { label: 'Feed URL', type: 'url' } },
  egress: ['config:feedUrl'],
  async sync(ctx) {
    // …return { documents, cursor }
  },
});

See also

• SDK Reference — how config, schedule, and the cursor surface inside sync(ctx)
• Cursors and Feeds — the feeds model and per-feed cursors
• Generated API reference — the exact types, from the shipped package
• Sources concept — source lifecycle and the no-credentials policy