Arsalan YasinNearly every SaaS team I speak with in 2026 has shipped an AI feature of some kind. Far fewer can tell me what a single answer from that feature costs them. That gap is where margin quietly disappears. A knowledge base that felt free during the prototype becomes a variable cost line that grows in lockstep with the customers you worked hardest to win.
I come at this from two directions. As an Odoo consultant in Australia businesses call when ERP numbers stop matching bank numbers, and as the person who has built retrieval augmented generation knowledge bases for platforms where the monthly bill had to stay under a figure already promised to a board. Both roles lead to the same discipline. Understand the unit economics of an AI feature before the invoice explains them to you.
The Quiet Margin Problem Behind Every AI Feature
Classic software economics were generous. Once the product was built, serving one more customer cost close to nothing, which is how the industry settled into gross margins of 70 to 80 percent. AI bends that curve. Inference cost is charged per token, per request, every time. Reported benchmarks through 2026 put AI heavy products closer to the low 50s, and gross margin compression has moved from analyst decks into board meetings.
The uncomfortable part is not that AI is expensive. It is that AI COGS scales with adoption. A feature everyone loves takes a real bite out of your blended gross margin, so the finance story gets worse at precisely the moment the product story gets better. Cost efficiency is not a frugality exercise. It is how you keep the option to grow.
Where the Money Actually Goes in an AI Knowledge Base
Split the bill into three layers, because they behave very differently.
Ingestion and embedding: Turning documents into vectors is mostly a one time cost, paid when content lands and amortised across every question it later answers. Your chunking strategy decides both how much you pay and how useful the result is, with 256 to 1024 tokens the usual working span.
Retrieval and storage: The vector database is rent. It scales with corpus size rather than traffic and is rarely the villain. Whether you run pgvector beside Postgres or pay for a managed service, it is usually a small fraction of the total.
Generation: This layer scales with success, and in most deployments I have reviewed it is most of the run rate. Every answer pushes a system prompt, a question and several retrieved chunks through the context window, then pays again for the tokens coming back. Token consumption, not model choice alone, is what you are buying.
The Numbers That Belong on Your Dashboard
Most teams measure AI spend as one undifferentiated invoice. Six metrics change that:
- Cost per request, tracked by feature rather than in aggregate
- Cost per active user, reported as a median and a top decile figure
- Cache hit rate across your semantic caching layer
- Average tokens per answer, split between retrieved context and generated output
- AI attach rate, the share of paying customers who actually touch the feature
- Blended gross margin, with AI revenue and subscription revenue calculated separately before they are combined
The top decile figure is the one that surprises people. Under flat pricing, a small group of power users consuming several times the median is quietly subsidised by everyone else.
Five Levers That Cut Spend Without Cutting Answer Quality
Route by query complexity: Most questions arriving at a knowledge base are simple lookups, and sending them all to your best model is like couriering every letter. Model routing sends routine retrieval to a small, cheap model and reserves the frontier model for reasoning heavy work. Which model you route to matters as much as how often you call it, and I worked through those trade offs in Gemini, Claude and GPT for Odoo development.
Cache semantically, not just literally: Users ask the same thing in different words, and a literal cache misses that completely. Semantic caching compares the query embedding against recent questions and returns a stored answer when similarity is close enough. In support style corpora a hit rate of a third is achievable, and every hit is an inference call you never pay for.
Fix chunking before you blame the model: Over retrieval is a quiet tax. Pulling ten chunks when three would do inflates the context window on every request, forever. Tightening the similarity threshold, lowering the retrieval count and adding a two stage retrieval pass with reranking usually improves answer quality while reducing spend.
Re-embed on change, not on a schedule: Reindexing an entire corpus nightly is a common and expensive habit. Track content hashes and re-embed only what moved. Save a full pass for a genuine embedding model upgrade.
Trim what you send: Prompt optimisation is unglamorous and effective. Boilerplate instructions, duplicated guidance and raw document metadata ride along on every request. Cutting a few hundred tokens from a prompt used ten thousand times a month is real money.
What This Means If You Run Odoo Alongside a SaaS Product
If your ERP and your product share a profit and loss statement, this is where the two worlds meet. Book AI provider invoices as cost of goods sold rather than operating expense. Folding them into opex flatters gross margin and hides the compression until it is structural. In Odoo, use analytic accounts and analytic distribution so vendor bills from your model provider and vector database are tagged to the product line they serve. A margin report by product line then stops being a spreadsheet exercise.
If you charge for the feature, the Subscriptions module handles a hybrid pricing model cleanly. A platform fee plus a metered allowance, with overage pricing above the threshold, aligns revenue with the cost you incur. Usage based pricing is not a growth hack here. It is what stops your best customers from becoming your worst margin.
A Budget Model You Can Run This Week
You do not need a forecasting tool, just one sheet of arithmetic with the assumptions written where a colleague can argue with them.
Monthly AI knowledge base budget
Active users on the feature 400
Questions per active user 18 -> 7,200 asked
Served from semantic cache 35% -> 4,680 billable
Routed to the small model 70% -> 3,276 calls
Routed to the frontier model 30% -> 1,404 calls
Small model 3,276 x $0.004 = $ 13.10
Frontier model 1,404 x $0.030 = $ 42.12
Vector store and storage = $ 70.00
Re-embedding (changed docs only) = $ 12.00
---------------------------------------------
Monthly run rate = $137.22
Cost per active user = $ 0.34
Cost per question asked = $ 0.019Every line is a lever. Move the cache hit rate up, shift the routing split, cut context size, and the run rate responds. You now hold a cost per active user you can set against what that user pays you.
If you are trying to work out where your AI spend is actually going, or how to get that picture into Odoo so your margin reports reflect it, that is a conversation I have often. Book a Consultation and we can walk through your cost structure, your routing and caching options, and the accounting setup that keeps those numbers visible month to month.
Conclusion
Cost efficiency in SaaS operations is not about spending less on AI. It is about knowing what an answer costs, pricing accordingly, and removing waste that delivers nothing to the user. The teams that get this right are not running weaker knowledge bases. They cache aggressively, route deliberately, retrieve precisely and account for AI COGS honestly. Their budget stays boring while the product keeps improving, which is where you want to be when usage takes off.
Frequently Asked Questions
What is a realistic gross margin for a SaaS product with an AI knowledge base?
Traditional SaaS sits at 70 to 80 percent and AI heavy products have been reported closer to the low 50s. Somewhere in the 60s is fair where AI is one capability rather than the whole value proposition. Calculate AI margin separately before blending it, otherwise the headline tells you nothing.
Should AI spend be booked as COGS or as operating expense?
As cost of goods sold. Inference cost, vector database hosting and embedding charges are direct costs of serving the customer. Booking them as opex inflates gross margin and delays the moment you notice compression. In Odoo, analytic distribution on those vendor bills keeps attribution clean per product line.
How much can semantic caching realistically save?
It depends on how repetitive your questions are. Support and onboarding corpora repeat heavily with hit rates around a third, while record specific queries cache poorly. Measure your real hit rate for two weeks before budgeting for the saving.
Do I need a dedicated vector database, or is pgvector enough?
For most knowledge bases below a few million chunks, pgvector beside your existing Postgres is enough, and it removes a service, a bill and a failure mode. Move to a dedicated vector database when retrieval latency at your real query volume becomes the constraint.
When should I charge separately for the AI feature?
When cost per active user stops being a rounding error against the subscription price, or when top decile users cost several times the median. A hybrid pricing model with an included allowance and usage based overage usually lands better than a sudden rise on the base plan.