Ingestors
The services that bring data into the platform. Each data source links to its ingestor through its method of ingestion.
There are three kinds:
- APIs — we call an official API and receive structured data.
- Scrapers — we read the source’s website and extract the data ourselves.
- Fetch scripts — batch scripts that download or collect a dataset once or on demand.
APIs
No match.
Served and persisting output
No endpoint — in-process or library
Output not persisted or not integrated
Data store
Active data intake
Found, not in the pipeline yet
Not documented yet
Scrapers
No match.
Served and persisting output
No endpoint — in-process or library
Output not persisted or not integrated
Data store
Active data intake
Found, not in the pipeline yet
Not documented yet
Fetch scripts
No match.
Served and persisting output
No endpoint — in-process or library
Output not persisted or not integrated
Data store
Active data intake
Found, not in the pipeline yet
Not documented yet