Read me first

If you have just been handed this project and you are not sure what it is, start here. This page assumes no technical background. It explains who the software is for, what it does, the world it belongs to, and what you would actually do with it — first on day one, and then week after week.

There is no code in this document. When you want the commands, the other pages under user-docs/ have them, and there is a map of those pages near the end.

---

1. Who this is for

This is built for the people who keep a company's AI features running and pay the bills for them. In practice that means three kinds of reader:

• The person responsible for the AI bill. Somebody has to answer "why did this cost what it cost last month?", and answer it per team or per feature rather than as one lump sum.
• The people who run the systems. Whoever gets paged when a feature stops working, and who needs it to keep working when an outside supplier has a bad afternoon.
• The people building the AI features. They want to send a request and get an answer, without each of them separately deciding which supplier to use, holding their own passwords for it, and rewriting their code when that choice changes.

It suits an organisation running AI features in earnest — several applications, more than one supplier, and a bill large enough that somebody has started asking questions about it. If you have exactly one small application talking to one supplier, this is more machinery than you need today.

It is not an AI model, and it does not answer questions itself. It has no consumer-facing screens; the only interface a human uses is an operator console.

---

2. What it does

Large language models — the AI behind things like chatbots, summarising and drafting — are rented from a handful of suppliers. Each has its own prices, its own strengths, its own bad days.

Normally each of your applications talks to a supplier directly. That means each one holds its own credentials, each one is written against that supplier's particular way of doing things, and each one falls over on its own when that supplier does.

LLM Router sits in the middle. Your applications send every AI request to it instead. For each request it decides which supplier and which model should answer, passes it on, returns the answer, and records what happened and what it cost.

Four things follow from that, and they are the whole point:

• Choice becomes a setting, not code. Changing which model your applications use is a configuration change in one place.
• A supplier outage stops being your outage. If one is failing, the request is sent to another that is working.
• Spending becomes visible while it happens, broken down by team, project and application — not discovered on an invoice weeks later.
• Limits can be enforced. Budgets and rate limits mean a runaway loop or an over-enthusiastic experiment cannot quietly spend a fortune.

---

3. The domain: renting intelligence by the word

A short orientation for readers coming from outside this world.

Modern AI text features are not built by training your own model. You rent access to someone else's, over the internet, and you are billed by volume — roughly by the amount of text going in and coming out. Every request costs a small amount of money, and the difference between models is large: one can easily be twenty or fifty times the price of another for the same request.

That produces a set of problems this software exists to solve:

• Price and ability pull in opposite directions. The best model is expensive and usually slower. The cheap fast one is fine for straightforward work and not for hard work. Sending everything to the best one is wasteful; sending everything to the cheap one produces poor results. The judgement is per request.
• Costs accumulate invisibly. Each request is fractions of a penny. Millions of them are a serious budget line, and by default nobody sees the total until it arrives.
• You are depending on somebody else's uptime. Suppliers have outages, rate limits, and slow spells. Depending on exactly one is a decision you will regret at some point.
• Where the data goes matters. Requests may contain personal information, and rules such as GDPR constrain where that can be sent. That has to be enforced by something, not left to each developer to remember.

The industry's answer to all four is a gateway — one door that every request goes through. This is one of those.

---

4. Where it fits

Upstream — your own applications. Anything that currently talks to an AI supplier. Because the router deliberately speaks the same language as the most common supplier's interface, existing applications usually need only to be pointed at a different address. Their code does not otherwise change.

Downstream — the AI suppliers. The build ships support for four: OpenAI, Anthropic, AWS Bedrock, and Google Vertex AI. You bring your own account with each one; the router uses your credentials on your behalf and adds no fee.

Alongside — the rest of your operations. It reports what it is doing in a standard format that common monitoring tools understand, so it can feed the dashboards and alerts you already have rather than demanding new ones.

Its own footprint. It runs on your infrastructure, with its own database. It is single-tenant: one installation serves one organisation. Your traffic and your spending data stay with you, and nothing about your requests is shared with anyone running a separate installation.

Signing in is handled by the shared platform identity service, so operators use the same account they use for the other tools in this ecosystem, rather than yet another password.

---

5. The very first thing to do

Run the initialisation, once. A fresh installation has an empty database — nothing to log into and nothing to route.

Initialisation clears the database and creates the minimum needed to start: your organisation, the first administrator, the default routing behaviour, and the basic reference data. It creates no example content: you begin with a clean, real system.

Two warnings worth taking seriously:

• It erases the target database. That is what makes it an initialisation. It will refuse to run over a database that already contains anything unless you explicitly force it, it makes you type the name of the environment you are about to initialise, and it demands extra confirmation for anything in the cloud. Those prompts are there on purpose — read them rather than clicking past them.
• Be certain which environment you are pointed at. You name the target explicitly every time. Check it before confirming.

Then sign in to the console and confirm it is running. production-setup.md walks through this step by step; it is the page to have open the first time.

---

6. Setting up a working environment

Once initialised, four things turn a blank installation into a working one. Roughly half a day, and mostly done once.

1. Connect your suppliers. Register each AI supplier you have an account with and provide your credentials for it. They are encrypted before being stored and are never displayed again afterwards.
2. Register the models you intend to use, with their prices. Prices are what the router uses both to choose between models and to calculate what you spent, so getting them right at the start is worth the ten minutes.
3. Set your routing policy — the standing instruction for how to choose. In plain terms: lean towards cheap, lean towards capable, lean towards fast, or balance them. You can start with the balanced default and adjust once you see real traffic.
4. Issue a key to each application, rather than one shared key for everything. This is the step people are tempted to skip, and the one that pays off most: keys are how spending is attributed, and how limits are applied. One key per application or team means the cost report answers real questions later. One key for everything means it cannot.

Then point your applications at the router instead of at the supplier, and set budgets and rate limits for the keys that need them.

---

7. Day to day

Most days, nothing needs doing. The router runs, and the console is something you glance at rather than sit in front of. The routine is:

• Look at the console. One screen: traffic and errors over the last day, spend so far, what each model is costing you, how each supplier is behaving, and where recent requests were sent. Ten seconds tells you whether today is normal.
• Notice the rerouted requests. Some requests do not go where they were asked to go — because a supplier was failing, or a policy sent them somewhere cheaper. The console marks these. A handful is the system doing its job. A sudden run of them means a supplier is having a bad day, and you knew before anyone complained.
• Answer cost questions as they arrive. Break spending down by team, project or application; look at a forecast of where the month is heading; get told when spend does something out of character. And when someone asks about one specific request, you can look up that individual request and see exactly what was decided for it and why.
• Adjust when the answer is wrong. If results are weaker than you would like, move the policy towards quality. If the bill is uncomfortable, move it towards cost, or set a firmer budget on whichever key is responsible.

---

8. Ongoing care

Not much, but not nothing.

• Monthly: check the prices. Suppliers change them, and add models. The router's arithmetic is only as good as the numbers you gave it, and stale prices mean both worse routing decisions and a misleading bill.
• Monthly: review the budgets. Teams grow, projects end. Limits set six months ago are usually wrong now.
• Regularly: back up the database. It holds your configuration and your entire cost history. Treat it like any other production database — your normal backup arrangements are what matter here, not anything specific to this software.
• As they age: rotate the credentials. Follow whatever policy you already have for supplier credentials; changing one here is a small operation.
• Occasionally: prune the history. Every request is recorded. That is what makes the reporting possible, and it grows steadily. Decide how far back you need to be able to look, and keep an eye on the size.
• When it matters: watch the error rate, not just the average response time. A supplier can look healthy on average while failing a slice of your traffic — and that slice is what your users notice.

---

9. A short glossary

Term	In plain words
LLM	Large Language Model. The AI that produces text.
Provider	A company you rent models from — OpenAI, Anthropic, AWS Bedrock, Google Vertex AI.
Model	One specific AI you can send a request to. Each has its own price and ability.
Gateway	Software sitting in front of something else, passing requests through. This is one.
Routing	Choosing which model answers a given request.
Failover / fallback	Automatically using another supplier when the first one fails.
Policy	Your standing instruction for how to choose — cheaper, better, faster, or balanced.
Virtual key	A key you issue to one of your own applications, so its spending can be counted and capped separately.
Budget	A spending cap. Requests beyond it are refused rather than billed.
Rate limit	A cap on how often something may ask, protecting you from runaway loops.
Token	The unit suppliers bill by — roughly a word-and-a-bit of text.
Semantic cache	Reusing an earlier answer when a new question means the same thing, even if worded differently. Off unless you turn it on.
Circuit breaker	Backing off from a supplier that is clearly failing, instead of retrying into a wall.
PII	Personal information. Its presence can change where a request is allowed to be sent.
Single-tenant	One installation serves one organisation. Yours is yours.

---

10. The other documents

Document	What it is for
getting-started.md	The fuller introduction, with worked examples. Read after this one.
production-setup.md	Preparing a real environment for first use — the day-one page.
useful-commands.md	The commands for running, deploying and operating it.
api-reference.md	For developers connecting an application to the router.
openapi.yaml	The same interface in machine-readable form, for generating code.
testing.md	What has been tested, in plain language, and the full list of tests.

---

11. Who can do what

Two roles, deliberately few:

• Administrator — configures suppliers, models, policies, keys and budgets. The first administrator is created during initialisation.
• Viewer — can see the console and the reports, but changes nothing. The right role for people who need the cost picture without the ability to alter routing.

Applications do not use these roles at all. They authenticate with their own issued key, which lets them send requests and nothing else — it grants no access to the console or to configuration.

---

12. The sample data, and starting clean

To make evaluation easy, the preview build ships pre-loaded with invented data — fictional traffic, spending and provider history, so every screen is populated and you can judge the software rather than stare at empty tables.

None of it is real. No genuine credentials, no genuine requests, no genuine costs. It exists so the console has something to show.

When you move to a real installation, do not carry it over. Run the production initialisation described in section 5, which starts you from empty. Demo data and real data should never share a database — mixed together, every cost report becomes fiction.

---

13. Questions people ask first

Does this make our AI cheaper? It can, by sending straightforward work to cheaper models and by letting you cap what is spent. It adds no fee of its own. But it does not negotiate your prices — you still pay your suppliers directly, at your rates.

Do we have to change our applications? Usually very little. The router deliberately accepts requests in the same shape as the most widely used supplier's interface, so for most applications it is a change of address rather than a rewrite.

What happens if the router itself goes down? It is in the path of your requests, so treat it as production infrastructure: run more than one copy and monitor it, as you would anything else your features depend on.

Can we keep using just one supplier? Yes. You still get the spending visibility, the limits, and one consistent interface. Add a second when you want failover.

Is our data being sent anywhere new? No. Requests go from your installation to the suppliers you configured, using your own accounts. The software runs on your infrastructure and keeps its records in your database.

Why does it look so plain? Because it is a core build — the functionality, without styling or branding. That is deliberate, and the next section explains what to do about it.

---

14. Where to get help

This project was generated entirely by AI, and what you have is a core build: it does the job, without visual polish or company branding. It is meant to be taken and made your own — restyled, rebranded, and extended with whatever your organisation actually needs.

If you have the domain knowledge and are comfortable working with AI tools, you can do that yourself. If you would rather work with someone, or you simply want to see how others have approached it, go to wbsp.ai. The community there has built many variations of this and other applications, and one may already be closer to what you want than starting from here. It is also the place to ask questions, suggest what this should do next, or find people who can build a bespoke version with you.