NVIDIA splits agent AI: cheap execution, expensive reasoning

NVIDIA has released Nemotron 3.5 Lightning — an open 30B mixture-of-experts model with roughly 3B active parameters — together with NeMo Switchyard, an open routing library. The message is blunt: long-running, always-on AI agents should not spend frontier-model tokens on every routine step.

This is not another chatbot launch. It is an architecture signal for enterprises already moving agents into production. Most of an agent loop is high-volume execution: tool calls, result validation, formatting, git operations, subagent delegation and status updates. When every one of those steps hits a frontier model, cost and latency become board issues, not just developer preferences.

What NVIDIA actually shipped

According to NVIDIA’s technical blog (11 August 2026), Lightning is built for the execution layer of always-on agents. The MoE design routes each token to only a few experts, combining larger-model capacity with small-model compute cost. NVIDIA claims up to 4x the output speed of similar-sized models, and up to 30% faster completion of agentic workloads on PinchBench versus Qwen3.6 35B at comparable accuracy (86% on PinchBench).

Weights, training data and recipes are released under OpenMDW-1.1. The package includes speculative decoding support (multi-token prediction, DSpark and DFlash), an NVFP4 checkpoint alongside BF16, and an open agentic RL dataset (Nemotron-RL Agentic Terminal Pivot) used for parts of coding-agent training. The model is positioned to run from local systems — Jetson, GeForce RTX, DGX Spark and DGX Station — through data center and cloud.

NeMo Switchyard is the second half of the release: an open library for intelligent routing inside existing agent tools. Planning and hard reasoning can go up to frontier models; high-volume execution can go down to Lightning — without rewriting the application. NVIDIA also points to a broad partner ecosystem across harnesses, inference stacks and cloud platforms.

CNBC corroborates the timing and strategy: Lightning is NVIDIA’s first open-model drop after CEO Jensen Huang publicly defended open-weight AI, arguing that “free AI” still drives GPU demand. The commercial logic is not charity. It is to make agent volume cheaper and more local — on NVIDIA’s stack.

Why leaders should care now

Many organizations have moved from pilot chatbots to tool-using agents in engineering, service, finance and security. That creates three immediate leadership decisions:

• Cost architecture. If most agent calls are small and only a minority are hard, pricing the whole fleet as frontier is waste. Lightning/Switchyard formalizes a pattern many teams already attempt ad hoc: cheap models for bulk, expensive models for exceptions. CFOs and CIOs need token cost per process, not only per vendor invoice.

• Control plane and lock-in. Once routing becomes a first-class component, someone must own policy: which model gets which task, under which data region, with which logs and kill switches. Switchyard is open, but it sits close to NVIDIA’s agent and GPU ecosystem. That can increase control — or create a new dependency if routing, telemetry and fine-tunes become hard to move.

• Local and sovereign operation. Lightning is explicitly positioned for edge and on-prem. For privacy-, resilience- or sector-regulated workloads, parts of the agent fleet can run without every tool call leaving controlled infrastructure.

CISO angle: faster agents, larger blast radius

Faster execution models raise productivity and misuse velocity if agents hold broad privileges. A model tuned for tool use and terminal work needs:

• strong identity and short-lived credentials per agent
• policy outside the prompt (what the agent is allowed to do, not only what it is asked)
• logging of tool calls, model choices and router decisions
• human approval for actions that touch money, production, identity or external communications
• evaluation of fine-tunes: domain adaptation can raise accuracy and quietly weaken safety refusals

NVIDIA itself notes harness-oriented training and partners such as CrowdStrike adapting Lightning for cybersecurity. That underscores the point: this is an execution engine for privileged workflows, not a neutral text toy.

Board questions for the next AI update

Skip “did we try the newest model?” Ask instead:

• What share of our agent tokens goes to planning versus execution today?
• Do we have an explicit routing policy, or does every team pick models ad hoc?
• Can we run the execution layer on-prem/edge where data requires it without losing observability?
• What is the patch, eval and rollback path when a cheap specialist model is fine-tuned on internal data?
• Who owns cost SLA and security SLA for multi-model agents — product, platform or security?

Competitive context

Lightning lands in a market where efficient open models already compete for agent volume: Qwen-class systems, Meta’s Muse Glimmer for local agents, and European platforms hosting third-party open weights under regional controls. NVIDIA’s differentiation is not only the model card, but the combination of open execution model, router, GPU stack and post-training ecosystem.

The practical lesson is simple: agent economics is now system design. Keep sending all agent traffic to one expensive model and you lose on cost. Unleash cheap execution models without a control plane and you lose on risk. Lightning and Switchyard make both outcomes more urgent — because they lower the barrier to always-on agent fleets at scale.

Sources and media

Primary source: NVIDIA Technical Blog, https://developer.nvidia.com/blog/nvidia-nemotron-3-5-lightning-delivers-fast-accurate-specialized-task-execution-for-long-running-agents/ NVIDIA Blog (product context): https://blogs.nvidia.com/blog/nemotron-lightning-switchyard-rtx-dgx/ Corroboration: CNBC, https://www.cnbc.com/2026/08/11/nvidia-releases-nemotron-3point5-lightning-open-source-ai-model-.html Official X post: https://x.com/nvidia/status/2087172614896988545 Thumbnail: OpenAI Image 2 / hogby.ai