How to Reduce LLM Costs with OpenSmartRoute
Reducing large language model (LLM) costs involves selecting cost-effective targets and optimizing request routing. OpenSmartRoute offers tools to manage and lower LLM expenses by controlling target selection, request objectives, and constraints.
Understanding OpenSmartRoute's Routing Capabilities
OpenSmartRoute functions as an LLM control plane, deciding who answers based on criteria such as cost, quality, and speed. It allows specifying objectives like minimizing cost while maintaining acceptable quality, and applying constraints such as maximum cost per request.
Routing Requests Efficiently
Requests are routed via the API endpoint, where parameters like objective and constraints guide target selection. For example, setting a lower cost weight in the objective trades off some quality for reduced expenses.
r = httpx.post(
"https://api.opensmartroute.ai/api/v1/route",
headers={"Authorization": f"Bearer {os.environ['OSR_API_KEY']}"},
json={"text": "Prove that sqrt(2) is irrational.", "top_k": 3, "objective": {"quality": 0.5, "cost": 1.0}}
)
print(r.json()["target"]["id"], r.json()["confidence"])
Declaring Targets for Cost Management
Targets are described with their costs, capabilities, and speed. By declaring targets with lower-cost models or deterministic tools, the router prefers cheaper options when quality permits.



