A new decoy counting benchmark exposes a gap between seeing and reasoning. Elorian’s Glimpse tries to bridge it by thinking with images.
Read the original at Latent Space: Can Frontier Models See Past a Decoy? Elorian Tests Visual Reasoning
How OpenSmartRoute helps
A published score is a starting point, not the answer for one team's prompts. 'osr eval' measures routing accuracy on the team's own data and can fail a build when it drops. The hosted platform adds public rankings built from real traffic, so a number can be checked against what the model does in use.
For a team that routes through the router, a release like this is one catalogue entry. The new model competes on the next request against what the team already runs, on quality, cost and speed. If it answers well it earns more traffic; if it fails it gets less, with no code change in the app.
Source: https://latent.space/p/elorian/