Workhorse LLMs: Why Open Source Models Dominate Closed Source for Batch Tasks

Summary

As LLMs become more prolific, we’ve noticed that teams still reach for closed sourced models like GPT, Claude, and Gemini for nearly every task. While this may have been the right call a year ago, teams today are unknowingly missing out on huge cost savings and performance gains by not considering open source alternatives.It is true that at the frontier of intelligence, the most powerful closed source LLMs dominate their open source counterparts. However, many common tasks for LLMs don’t require PhD level reasoning. Instead, they require a workhorse LLM: something that’s reliable for low-to-medium difficulty tasks – things like classification, summarization, and data extraction.Not only are there suitable replacements for closed source workhorses like GPT-4o-mini, but the equivalent replacements are often less expensive and more intelligent. When latency isn’t an issue, open source models have an even larger edge for cost savings if running jobs in bulk through a batch inference provider like Sutro.In this guide, we compare the performance and cost of workhorse models. After the analysis, we provide a handy conversion chart that helps you pick the best open source replacement for the closed source models you may have used along with the cost savings you should expect by making the switch.How Do Open Source LLMs Stack Up Against Closed Source LLMs?A common question organizations ask us is how the cost and performance of open source models stack up against closed source models. To answer this question, it’s best to divide the field into two categories: frontier models and workhorse models.Frontier models are the biggest, most capable models. They promise emergent capabilities, generalization across tasks, and the ability to handle complex context and instructions.At the frontier, closed source models like Claude Opus 4.0, OpenAI’s o3 model, and Gemini 2.5 Pro dominate. However, open source models like Qwen 3 235B-A22B and DeepSeek R1 (and the to-be-released R2) are cl...