Building block · Secure inference
Secure inference, no third-party model egress.
Call a model directly with your own prompt, or let RAG answer from your own data. Runs inside the Vectros perimeter either way, your content never leaves for a third-party model, under the same access-scope enforcement as everything else on the platform. The exception is a call you route to a model provider you configure: it is processed where that provider processes it and leaves the Vectros perimeter.
The alternative
What you'd otherwise build.
Calling a model is the easy part. Knowing exactly where your content goes is what takes the work.
- Route your own calls to a model vendor. Pick a provider, integrate their SDK, and manage the relationship yourself.
- Work out a compliance posture with them. Negotiate what their side of the data-handling story looks like, on your own.
- Decide, per request, what is allowed to cross the boundary. Without a platform default, that judgment call sits on you every time.
- Build your own metering and region handling. Token and cost tracking, plus your own answer to which region each request is served from.
What's actually different
How region and cost work.
Region and cost
Data is stored in a US region. Inference is served from a US region by default; a global region is opt-in under a signed waiver.Pricing: the default region typically carries a premium that is a complete pass-through of the underlying cloud provider's own cross-region cost, not markup, and the global region is billed at a lower rate.
Same enforcement as everywhere else
The same access-scope enforcement gates every call here, same as every other building block on this site: not a separate service credential to manage.
Isolate your own clients, orgs, or anything else
Give a caller access to exactly one client, one org, or any other axis your product needs. Its inference calls, and anything they ground on, can never reach outside that boundary, no matter what the call itself says: leave the scope off and the request is rejected outright, not silently widened. Org and client are two reserved names, registered the same way as anything else you define.
// clientId is the id of a client entity you already created.
const { token } = await client.auth.mintToken({
scope: {
allowedActions: ['inference:r', 'search:r', 'documents:r'],
dataScope: { 'scope:client': [clientId] },
},
});
const scoped = new VectrosClient({ token, environment: 'production' });
// Grounding retrieval stays inside the same boundary as the call itself.
const stream = await scoped.inference.ragInference({
query: 'What follow-up was scheduled?',
search: { mode: 'HYBRID', limit: 10, scope: 'client:' + clientId },
});See the full access-control mechanics: roles, credentials, audit →
Chat is stateless: there is no managed conversation history. Resend your own message history each turn.
See it in a real app
Already running underneath what you've seen elsewhere.
This is the layer RAG is built on. Call it directly here, without retrieval, for your own prompts.
This is one of five building blocks Vectros is built from.
See how it all fits together on the platform page.