↓ Skip to main content
Inês Garcia

Open Source Won the AI Agent War — Here's What That Means for Data Teams

·2 mins
Open Source AI Agents Data Science LLMs
Table of Contents

In January 2024, Hugging Face published a benchmark that most people in the data world missed. They compared open-source LLMs against GPT-3.5 and GPT-4 on agent tasks — using a dataset that requires web search and calculator use, the fundamentals of any analytics agent.

The result: Mixtral-8x7B beat GPT-3.5 on agent tasks, out of the box, with no fine-tuning.

This is the moment open source won the AI agent war for data teams. Here’s why it matters.

The Benchmark Details
#

The HuggingFace team created a dataset combining:

  • HotpotQA: multi-hop questions requiring combining information from multiple sources
  • GSM8K: grade-school math requiring precise calculation (not estimation)
  • GAIA: hard general AI assistant tasks requiring multiple steps

These map well to analytics tasks: multi-source data questions, precise numeric calculations, and complex multi-step investigations.

Results:

GPT-4:              ~87%
Mixtral-8x7B:       ~77%  ← beats GPT-3.5
GPT-3.5:            ~75%
OpenHermes-2.5:     ~60%
Llama2-70b:         ~45%

Key finding: Mixtral is within 10 points of GPT-4 on agent tasks, and surpasses GPT-3.5, without any agent-specific fine-tuning. With fine-tuning — which HuggingFace explicitly recommends — the gap narrows further.

Why Open Source Matters for Data Teams
#

For most data teams, the reason to care about open source is not political. It’s practical:

Privacy. Sending your company’s query logs, financial metrics, or user behavior data to OpenAI’s API is a meaningful data governance decision. Running Mixtral locally — or on an inference provider like Cloudflare Workers AI — means the data never leaves your infrastructure.

Cost at scale. An analytics agent running 500 queries per day against GPT-4 costs ~$300/month. The same agent on Workers AI with Llama 3.1 70B costs ~$15/month. For production workloads, this is a real constraint.

Customization. Open-source models can be fine-tuned on your domain. A Mixtral fine-tuned on your specific metric definitions and query patterns will outperform a generic GPT-4 call on your specific tasks.

The Practical Recommendation
#

For a data team building an analytics agent today:

  1. Start with Workers AI, Together AI, or Groq for fast, cheap, private inference
  2. Use Mixtral-8x7B or Llama 3.1 70B as your base model
  3. Fine-tune on 50–100 examples of your specific query patterns — this is what the HuggingFace team says would push Mixtral past GPT-4
  4. Evaluate every change against your test dataset

The era of “we need GPT-4 or it doesn’t work” is over for most analytics use cases. Open source is here. The question is whether your team is ready to use it.