Building block · Secure inference
Secure inference, no third-party model egress.
Call a model directly with your own prompt, or let RAG answer from your own data. Runs inside the Vectros perimeter either way, your content never leaves for a third-party model, under the same access-scope enforcement as everything else on the platform.
The alternative
What you'd otherwise build.
Calling a model is the easy part. Knowing exactly where your content goes is what takes the work.
- Route your own calls to a model vendor. Pick a provider, integrate their SDK, and manage the relationship yourself.
- Work out a compliance posture with them. Negotiate what their side of the data-handling story looks like, on your own.
- Decide, per request, what is allowed to cross the boundary. Without a platform default, that judgment call sits on you every time.
- Build your own metering and residency logic. Token and cost tracking, plus your own answer to where a request actually gets served, if you need to guarantee that at all.
What's actually different
Residency is a request, not an afterthought.
Residency, priced at cost
Default to guaranteed US-only serving, or opt into the lower-cost global-region rate once entitled. That US premium is a complete pass-through of the underlying cloud provider's own cross-region cost, not markup.
Same enforcement as everywhere else
The same access-scope enforcement gates every call here, same as every other building block on this site: not a separate service credential to manage.
Isolate your own clients, orgs, or anything else
Give a caller access to exactly one client, one org, or any other axis your product needs. Its inference calls, and anything they ground on, can never reach outside that boundary, no matter what the call itself says: leave the scope off and the request is rejected outright, not silently widened. Org and client are two reserved names, registered the same way as anything else you define.
// clientId is the id of a client entity you already created.
const { token } = await client.auth.mintToken({
scope: {
allowedActions: ['inference:r', 'search:r', 'documents:r'],
dataScope: { 'scope:client': [clientId] },
},
});
const scoped = new VectrosClient({ token, environment: 'production' });
// Grounding retrieval stays inside the same boundary as the call itself.
const stream = await scoped.inference.ragInference({
query: 'What follow-up was scheduled?',
search: { mode: 'HYBRID', limit: 10, scope: 'client:' + clientId },
});See the full access-control mechanics: roles, credentials, audit →
Chat is stateless: there is no managed conversation history. Resend your own message history each turn.
See it in a real app
Already running underneath what you've seen elsewhere.
This is the layer RAG and the platform's agentic capabilities are already built on. Call it directly here, without retrieval, for your own prompts.
This is one of five building blocks Vectros is built from.
See how it all fits together on the platform page.