Adding AI Features to a Web App Without Wasting Money

Every product now has an "add AI" item on the roadmap, and most of them ship a chat bubble nobody opens. The features that get used are narrower, more specific, and usually invisible as AI features at all.

Where AI actually earns its place

The pattern that works: use a model where the task is fuzzy, the input is messy, and a wrong answer is cheap to correct.

• Turning unstructured text into structured data — parsing a pasted invoice, a CV, a support email
• Summarising long content people would otherwise skim badly
• Drafting a first version — a product description, a reply, a title — that a human then edits
• Semantic search, where users search by meaning rather than exact keywords
• Classification and routing — tagging tickets, detecting intent

Where it does not earn its place: anything requiring exact arithmetic, anything where a confident wrong answer is expensive, and anything a database query already answers.

Never call the API from the browser

This is the most common security mistake in AI features. Putting your key in a NEXT_PUBLIC_ variable exposes it to anyone who opens devtools, and people scrape for these.

Always route through your own backend:

// pages/api/summarise.js
export default async function handler(req, res) {
  if (req.method !== 'POST') return res.status(405).end();

  // Authenticate and rate-limit BEFORE spending money
  const user = await getUser(req);
  if (!user) return res.status(401).json({ error: 'Unauthorised' });
  if (await overQuota(user)) return res.status(429).json({ error: 'Limit reached' });

  const result = await callModel(process.env.MODEL_API_KEY, req.body.text);
  res.json({ summary: result });
}

Rate limiting is not optional. An unauthenticated AI endpoint is a bill waiting to happen, and it will be found faster than you expect.

Controlling cost

You pay per token, in and out. The levers that matter, roughly in order:

1. Use a smaller, faster model for simple tasks. Classification and extraction rarely need your most capable model.
2. Trim the input. Sending an entire document when three paragraphs would do is the most common source of waste.
3. Cap the output length explicitly.
4. Cache aggressively. Identical inputs should not be paid for twice — hash the input and store the result.
5. Set hard per-user daily quotas from day one, not after the first surprise invoice.

Prompts are code

Treat them accordingly:

• Keep them in version control, not scattered inline as string literals
• Be explicit about the output format. If you need JSON, say so and specify the shape.
• Give one or two examples — this improves consistency more than lengthy instructions
• Always validate what comes back. Models sometimes return malformed JSON or extra prose.
• Version prompts so you can tell whether a behaviour change came from your edit or from a model update

Handling failure

The API will be slow, rate-limited or down at some point. Design for it:

• Timeouts, so a hung request does not hang your route
• Retry once with backoff, then give up — do not retry forever
• A fallback path: show the raw content instead of the summary, keep the manual form available
• Stream responses where the output is long, so the user sees progress instead of a spinner
• Log inputs and outputs so you can debug complaints later

Retrieval, when the model needs your data

Models do not know your database. When users ask about your content, the standard approach is retrieval-augmented generation: find the relevant documents first, then include them in the prompt.

The quality of a RAG feature is almost entirely determined by retrieval, not by the model. If the right document is not fetched, no model can rescue the answer. Spend your effort on chunking and search quality before you spend it on prompt wording.

Be honest in the interface

• Label generated content as generated
• Make it easy to edit rather than accept blindly
• Do not present a confident tone as a guarantee of accuracy
• Tell users what happens to their data

Trust is the actual product constraint. One confidently wrong output that a user acted on costs more than the feature was ever worth.

Start small

Pick one narrow task where the current experience is tedious, ship it behind a flag, measure whether anyone uses it, and expand only if they do. That sequence produces features people keep; the chat bubble approach produces features people close.