AI News · Models ·
OpenAI opens its fast Decisions API model to all customers
OpenAI made its Decisions API model available to all customers on Thursday, adding voice and visual inputs and regional inference to cut latency. The model is a reconfigured version of ChatGPT Luna that responds in 100 to 180 milliseconds, according to OpenAI API product lead Nikunj Handa. It competes with TypeSafe's Jev in the growing category of decision models.
Key points
- OpenAI's Decisions API model is now available to all customers, with voice and visual inputs.
- OpenAI says responses take 100 to 180 milliseconds, down from one to three seconds.
- Box, Datadog, Ramp and Speak tested the model in early trials.
What happened: OpenAI made its Decisions API model widely available to all customers on Thursday, TechCrunch reported. The model was announced at the company's developer day on September 29. The public version adds voice and visual modalities, and OpenAI has placed inference resources in different regions to reduce latency.
The details: Decision models, sometimes called System One models, choose quickly among options and return a confidence rating. TypeSafe's Jev created the category, and developers have used such models to replace LLMs in many workflows at lower cost. Rather than train a dedicated model, OpenAI reconfigured ChatGPT Luna for faster, cheaper inference. Nikunj Handa, product lead for OpenAI's APIs, said response times fell from one to three seconds to 100 to 180 milliseconds. Engineers at Box, Datadog, Ramp and Speak tested the API, and Speak CTO Andrew Hsu said it cut the time between actions from 1.4 seconds to 0.13 seconds.
Background: Rankings vary. The AIM System One Index places OpenAI's model eighth and the latest Jev seventh, while the Artificial Analysis Decision Index places them fourteenth and third. Both indexes favor Liquid's D1 Ultra and Base Lab's Elephant-1. TypeSafe CEO Diogo Almeida urges users to test models on their own work.
What to watch: Handa said OpenAI hopes to reach about 60 milliseconds, which could allow an API call for every video frame. Pricing for the model was not reported.
Our take
Teams running high-volume classification or routing steps on full LLMs now have another low-latency option, but should test it on their own workloads because benchmark rankings disagree.