What is “gpt-oss” by OpenAI: How to get it for Free?

OpenAI has just announced a groundbreaking release that marks a significant shift in the landscape of artificial intelligence: the launch of gpt-oss-120b and gpt-oss-20b. These are OpenAI’s first open-weight language models since GPT-2, providing developers and enterprises with unprecedented flexibility and control over how they run, adapt, and deploy OpenAI’s sophisticated models. This move ushers in a new era where powerful AI is more accessible and customizable, balancing state-of-the-art performance with cost-effective deployment.

What Makes gpt-oss So Special?

The gpt-oss models distinguish themselves through their open-weight nature, exceptional performance, advanced architectural design, and a strong commitment to safety and deployment flexibility.

Unlike models available solely through APIs, the open-weight nature of gpt-oss provides full transparency and control. This allows developers to:

• Fine-tune models using parameter-efficient methods like LoRA, QLoRA, and PEFT. This means you can splice in proprietary data and ship new checkpoints in hours, not weeks.
• Optimize for specific environments by distilling or quantizing models, trimming context length, or applying structured sparsity to meet strict memory requirements for edge GPUs and even high-end laptops.
• Inspect attention patterns for security audits, inject domain adapters, retrain specific layers, or export to ONNX/Triton for containerized inference.
• Effectively treat these models as “programmable substrates,” allowing for deep customization and pushing the boundaries of AI applications.

For decision-makers, gpt-oss offers competitive performance without black boxes, providing greater control and flexibility across deployment, compliance, and cost considerations.

Performance That Rivals Proprietary Models

The gpt-oss models are not “stripped-down replicas” but are designed for real-world deployment and deliver impressive capabilities:

• gpt-oss-120b: This reasoning powerhouse boasts 117 billion total parameters, activating 5.1 billion parameters per token with architectural sparsity. It achieves near-parity with OpenAI o4-mini on core reasoning benchmarks and matches or exceeds o4-mini on competition coding, general problem-solving (MMLU, HLE), and tool calling (TauBench). Remarkably, it even outperforms o4-mini on health-related queries (HealthBench) and competition mathematics. This powerful model is efficient enough to run on a single 80GB GPU.
• gpt-oss-20b: This tool-savvy and lightweight model has 21 billion total parameters, activating 3.6 billion parameters per token. It delivers similar results to OpenAI o3-mini on common benchmarks and matches or exceeds o3-mini on the same evaluations, even outperforming it on competition mathematics and health. It is optimized for agentic tasks like code execution and tool use and can run efficiently on various hardware, including edge devices with just 16GB of memory.

Both models demonstrate strong performance on tool use, few-shot function calling, and Chain-of-Thought (CoT) reasoning.

Intelligent Reasoning with Chain-of-Thought (CoT)

The gpt-oss models support full Chain-of-Thought (CoT), enabling them to decompose complex queries into intermediate steps, similar to how humans reason. This capability is crucial for complex tasks and is what makes “gpt-oss” unique among other open-source models. Developers can easily adjust the reasoning effort (low, medium, or high) via the system message, allowing for a trade-off between latency and performance.

Crucially, OpenAI did not put any direct supervision on the CoT for either gpt-oss model. This is considered vital for monitoring model misbehavior, deception, and misuse, and it empowers developers and researchers to implement their own CoT monitoring systems. However, developers should not directly show CoTs to users in their applications, as they may contain hallucinated or harmful content.

Flexible Deployment: From Cloud to Edge

The gpt-oss models are designed for flexible and easy deployment anywhere – locally, on-device, or through third-party inference providers.

• gpt-oss-120b can be run on a single enterprise GPU in the cloud via platforms like Azure AI Foundry.
• gpt-oss-20b can be run locally on various Windows hardware, including discrete GPUs with 16GB+ VRAM. It’s available on Windows AI Foundry (and soon MacOS) via Foundry Local and can also be deployed on devices with Snapdragon processors through platforms like Hugging Face and Ollama.
• This enables cloud-optional deployment, allowing data to remain on-device in offline or secure network settings.

Both models will soon be API-compatible with the ubiquitous Responses API, making it easy to swap them into existing applications with minimal changes. OpenAI has also partnered with leading deployment platforms like Azure, Hugging Face, AWS, and hardware leaders like NVIDIA and AMD to ensure broad accessibility and optimized performance.

Advancing AI Democratization

By lowering barriers for emerging markets, resource-constrained sectors, and smaller organizations that may lack the budget or flexibility for proprietary models, gpt-oss helps expand “democratic AI rails”. This initiative, along with providing developer tools like the open-sourced o200k_harmony tokenizer and harmony renderer, underscores OpenAI’s commitment to democratize AI and offer a diverse portfolio of models.

The gpt-oss models have undergone comprehensive safety training and evaluations, including testing an adversarially fine-tuned version under their Preparedness Framework. They perform comparably to OpenAI’s frontier models on internal safety benchmarks.

In essence, gpt-oss is not just a release of new models; it’s an invitation to the global community to experiment, collaborate, and push the boundaries of what’s possible with AI. We look forward to seeing the incredible innovations that will emerge from this new chapter in open AI development.

Key Takeaways:

• OpenAI releases gpt-oss-120b and gpt-oss-20b, their first open-weight language models since GPT-2.
• gpt-oss models offer unprecedented customization and control for developers and enterprises.
• These models deliver impressive performance, rivalling proprietary models, with flexible deployment options from cloud to edge.
• gpt-oss supports Chain-of-Thought reasoning for complex tasks and allows for developer-controlled monitoring systems.
• The release aims to democratize AI by lowering barriers for wider adoption and fostering innovation.

Join our community by subscribing to our Weekly Newsletter to stay updated on the latest AI updates and technologies, including the tips and how-to guides. (Also, follow us on Instagram (@inner_detail) for more updates in your feed).

(For more such interesting informational, technology and innovation stuffs, keep reading The Inner Detail).