OpenAI recently introduced a new API setting, Flex processing. This capability provides developers with a more cost-effective means to manage less-critical AI-related workloads. This new capability is designed to reduce operational costs while supporting specialized workloads that will not benefit from ultra-fast response times. Flex processing is now available in beta for OpenAI’s o3 and o4-mini reasoning models.
Flex processing dramatically reduces the overall costs of using the API. As a result, it’s a very compelling place to run model training evaluation, data enrichment and other async workloads. The pricing structure reflects this cost-efficiency, offering o3 Flex processing at $5 per million input tokens and $20 per million output tokens. The regular processing rates for O3 ( https://www.openai.com/pricing ) are $10 per million input tokens. For output tokens, the rate jumps up to $40 per million. This pricing change means developers using Flex processing will be able to cut API costs by 50%.
It’s important for users to understand the trade-off that comes with lower prices. You’re going to feel it in your response time. Flex processing will at times face unavailability of resources. This imbalance creates a drag on the return on investment, even for mission-critical tasks it performs well. Those slower response times are especially well-suited for non-production use cases where real-time results are not a top priority.
OpenAI has tiered access to this new processing option according to usage tiers. Developers in all 3 tiers of OpenAI’s developer hierarchy must undergo ID verification. Only upon confirmation will they receive access to the o3 model with Flex processing. This user verification requirement adds a layer of security and accountability, contributing to OpenAI’s mission of promoting the safe and responsible use of AI technology.
Leave a Reply