Keeping LLM costs predictable when my users pay in gourdes

Question

AI APIs bill me in USD per token, but my users pay small amounts in HTG through MonCash. One heavy user could cost more than they pay. How do I keep the economics sane?

Accepted Answer

Treat AI usage like a metered resource with **budgets at three levels**.

**1. Per request: cap the size**
- Set `max_tokens` on every call, and limit input length (truncate long pastes, summarize old chat history).
- Use the **smallest model that passes your quality tests** for routine tasks. Reserve bigger models for hard ones (routing).
- Use **prompt caching** for long, repeated system prompts and documents. It cuts input cost substantially.

**2. Per user: quotas tied to what they paid**
~~~ts
const PLAN_QUOTA = { free: 20, basic: 300, pro: 1500 }; // AI requests per month

async function checkAiQuota(user: User) {
  const used = await usage.countThisMonth(user.id);
  if (used >= PLAN_QUOTA[user.plan]) throw new QuotaError("Ou rive nan limit mwa a. Ogmante plan ou oswa tann mwa pwochen.");
}
~~~
Record **actual tokens** from every API response, per user, so you know each plan's real cost.

**3. Global: a spending circuit breaker**
- Set a **monthly spend limit** in the provider's console.
- Add an app-level daily budget. When it's exceeded, switch to degraded mode (cheaper model or AI off) and alert yourself.

**Price in HTG with margin.** Compute the cost per active user from real usage data, convert at a conservative exchange rate, and add margin for currency swings. Revisit monthly, because both model prices and the exchange rate move.

1 Repons