Rajan Verma

On-prem inference · Demo

Your model. Your machine. Public LLM chat tools never see the data.

I put an LLM on your GPU — CPU if that is the machine — so your product can call a model you own. Your engineer gets a private URL, a token only your app holds, and a written handoff they can run without me. The screen stays yours.

How the install works See live examples

How the install works

  1. 01 Data stays on your server

    Tickets, policies, and mail never leave your machine for a hosted model API. Privacy is the architecture, not a checkbox.

  2. 02 I install serving where the files already are

    I bring up the stack on that machine — vLLM when you have GPU, Ollama when you do not. HTTPS in front. Nobody trains on your data.

  3. 03 You receive an API, not a chatbot company

    POST /v1/generate, GET /health, Bearer token. Same shape as a typical hosted generate call — except the machine is yours.

  4. 04 You create and rotate the token

    Nobody logs into the model. Only your app holds the secret.

  5. 05 I hand the machine to you in writing

    Runbook, health probe, how to stop the process. Your engineer is not stuck with my laptop.

  6. 06 Your engineer integrates — one POST

    A few lines in your product. The three examples below are fictional props so you can see the click. They are not the SKU.

What you get

A token site, paste-ready snippets, and a written handoff

After the install your engineer is not decoding my laptop. They mint and rotate the secret, copy a few lines into your product, and keep a one-page runbook — URL, health probe, how to stop the process. The demo is a working mock: fake names, fake token, same click.

Open the token demo

Live examples

Example 1 · live · draft

Harbor Desk

A small insurance agency lives in a shared inbox. The same three shapes keep arriving: a claim after an accident, a double charge on an invoice, a prospect asking for a homeowners quote. Today an agent opens the thread, rereads the policy context, and types a careful reply by hand. The queue waits. Nothing here is a real agency or a measured “hours saved” number.

With on-prem inference, their product POSTs that thread to a generate API on their server and gets a draft back. The agent still edits and hits send. Public LLM chat tools never see the policy number. Open the live inbox: sample mail is locked; Generate draft runs here.

Live Harbor inbox

Example 2 · live · classify

Fieldline CRM

A field-service CRM stores messy job notes a tech types on a phone: icing condenser, pilot that will not stay lit, a callback after a leak. The office needs two things from that blob — a tag (urgent / parts / done) and a short SMS the customer can actually read — without pasting the note into a public LLM chat tool. Fictional product. Not a real trade company.

Same generate API as Harbor. Different prompt, different UI. Their CRM POSTs the note; the model returns on their machine. Open the live jobs list: notes are locked; Generate runs here.

Live Fieldline notes

Example 3 · live · extract

Ledgerbox

A bookkeeping product needs structured fields, not a reply. Someone forwards a messy invoice: vendor name, amount, due date, invoice id buried in a paragraph. Today a bookkeeper copies those into the ledger by hand and hopes the decimals survive. Fictional product. Not a real firm.

Again the same POST. The prompt asks for labeled fields instead of a letter. Their app stores the result. That is still on-prem inference — classify, draft, or extract, whatever their product already shows on screen. Open the live extract: sample text is locked; Generate extract runs here.

Live Ledgerbox extract

Contact

Let’s put generate on your machine

If tickets, mail, or policies cannot leave your server, write. Tell me the machine — GPU or CPU — whether HTTPS is already in front, and the date you need the endpoint live. I come back with a URL, a token, /health, and a written handoff. You keep the product UI.

Public examples call a locked demo path with fixed sample text. Your engineer still uses a Bearer token on the real generate API.