List of Best
Sign in
LLMs

Together AI Tested Two Coding Models — and Found a Way to Cut Costs by Over Half

· Together AI Blog · 2 min read
Table of contents

On August 18, 2026, Together AI published a benchmark comparing two coding models: GPT-5.6 Sol and DeepSeek V4 Pro 0813. If you build AI coding agents, this test offers a practical lesson in trading speed for cost.

The numbers behind the test

Together AI ran 904 rollouts across 113 DeepSWE tasks, giving each model four trials (452 rollouts per model). The data shows clear trade-offs:

  • First-try accuracy: GPT-5.6 Sol led pass@1 with 72.7%, while DeepSeek V4 Pro 0813 scored 62.8%.
  • Four-try accuracy: DeepSeek took the lead on pass@4 with 88.5%, beating Sol’s 85.8%.
  • Rollout cost: DeepSeek cost just $0.24 per rollout, compared to $8.37 for Sol.
  • Speed and steps: Sol finished in a median time of 17 minutes and 53 steps. DeepSeek took 35 minutes and 146 steps.
  • Token usage: Sol used 59k output tokens, while DeepSeek consumed 101k.

Sol is fast. DeepSeek is cheap.

Why routing beats picking one model

Because the models have different strengths, Together AI tested a “Pro-first” routing cascade. The system runs DeepSeek first and escalates to GPT-5.6 Sol only if tests fail.

This routed setup solved 83.0% of tasks at an average cost of $3.35. That beats Sol’s solo score of 72.7% while cutting the $8.37 cost by more than half. Smart routing saves cash.

What you should keep in mind

These numbers come from Together AI’s own test environment — nobody outside the company has independently verified them yet. Together AI also says DeepSeek V4 Pro includes US hosting, a 1.05M context window, function calling, JSON mode, and an OpenAI-compatible API. Check these specs on your own account before making a pipeline change.

How to test this for your team

Do not rely on vendor benchmarks alone. Test both models against your private codebase:

  • Pin your model IDs and keep test conditions constant.
  • Run single-try and best-of-four evaluations on your tasks.
  • Test a Pro-first cascade against running Sol alone.
  • Track pass rates, wall time, token counts, and costs.

Always have humans review the code patches. A green test does not guarantee good code.

More in LLMs

← All news