Building block · Document ingestion
Ingest documents, indexed for search as they land.
Send text inline, or upload a file and get a presigned URL. Either way, it's searchable within moments, in full-text, semantic, or hybrid mode.
The alternative
What you'd otherwise build.
Getting unstructured content into something searchable is usually its own project, not a side effect of storing it.
- Extraction per file type. PDFs, images, spreadsheets, transcripts: each one its own extraction step to write and maintain.
- A chunking and embedding pipeline. Split, embed, store, and keep the split points from cutting a sentence in half at the worst possible spot.
- An async job queue. Ingest is not instant. Something has to track status, retry a failed extraction, and tell the caller when it is safe to search.
- Metadata bolted on after the fact. Your own layer to attach structured fields to a document, because the extraction step only gives you a blob of text.
- A second access-control surface for documents. Records and documents usually live in different stores, so the same authorization logic gets rewritten, and can quietly drift, for whichever one holds the file.
What's actually different
Every document can carry real, structured metadata.
Real structured metadata, not just text
Attach a schema to what you ingest and its metadata is validated and lookup-indexed the same way a record's fields are: client name, case type, channel, whatever you declare. Searchable and filterable alongside the extracted text, not just a blob.
Sensitive fields never reach the index
Mark a metadata field sensitive and it's excluded from the search index at write time, the same protection your records get. Nothing a query could surface, under any scope.
// Inline text, indexed as soon as it lands:
const note = await client.documents.ingestDocument({
body: {
title: 'Q3 support call transcript',
text: '...',
indexMode: 'HYBRID',
schemaId: 'support_call',
payload: { customer: 'acme-hr', channel: 'phone' },
},
});
// A file instead: request a presigned URL, then PUT the bytes yourself.
const upload = await client.documents.uploadDocument({
fileName: 'q3-support-call.pdf',
fileType: 'application/pdf',
indexMode: 'HYBRID',
});
await fetch(upload.uploadUrl, {
method: 'PUT',
body: fileBytes,
headers: { [upload.requiredHeaderName]: upload.requiredHeaderValue },
});
// Extraction and indexing happen after the PUT; poll for status when you need to know.Pay only for what's indexed: mark a document NONE and it's stored and retrievable, with zero search cost.
Isolate your own clients, orgs, or anything else
Give a caller access to exactly one client, one org, or any other axis your product needs. It can never read or write documents outside that boundary, no matter what the call itself says: leave the scope off and the request is rejected outright, not silently widened. Org and client are two reserved names, registered the same way as anything else you define. See the database building block for the same mechanics on records.
// clientId is the id of a client entity you already created.
const { token } = await client.auth.mintToken({
scope: {
allowedActions: ['documents:r', 'documents:c'],
dataScope: { 'scope:client': [clientId] },
},
});
const scoped = new VectrosClient({ token, environment: 'production' });
// Every read through it must name that same boundary, or it's rejected outright:
const theirDocs = await scoped.documents.listDocuments({
scope: 'client:' + clientId,
});
// A write through it must state ownership too, or it's refused the same way:
await scoped.documents.ingestDocument({
body: {
title: 'Intake call transcript',
text: '...',
indexMode: 'HYBRID',
scopes: ['client:' + clientId],
},
});See the full access-control mechanics: roles, credentials, audit →
The moment a document lands, it's queryable. See search for a ranked list, or RAG for a cited answer instead.
See it in a real app
Every case gets its own folder, scoped like everything else in it.
The RAVV reference app creates a case as a structured record with its own folder in one step. Every document a team member later uploads onto that case lands in that same folder, carrying the same client and org scope as the case itself. Second Brain's note-dump is the lower-stakes version of the same idea.
Read: the RAVV stack →This is one of five building blocks Vectros is built from.
See how it all fits together on the platform page.