01 Apr 2024 AI Engineering 5 min read

RAG-Tuned Models: Command R and Grounded Generation

Cohere's Command R, released 11 March 2024, treats grounded generation as a trained capability rather than a prompt trick: span-level citations, native tool use, 128k context, and open research weights. We examine what a RAG-tuned model changes for production systems, what it does not solve, and where the category is heading.

The Grounding Problem in Production RAG

Retrieval-augmented generation is not new. Lewis et al. formalized the pattern in 2020: fetch relevant documents at inference time, condition the model on them, generate. In practice, most deployments through 2023 implemented grounding as prompt engineering. Retrieved chunks were pasted into the context window with an instruction to answer only from the sources. Whether the model complied was unverifiable. Attribution, where it existed at all, was a regex over free text. The model was never trained for the job it was given.

For production systems this is a structural weakness. An answer that cannot be traced to a source cannot be audited, and an audit trail is precisely what enterprise users of RAG systems ask for. On 11 March 2024, Cohere released Command R: a model in which grounding is a trained capability with a defined API contract. That shift — grounded generation as a product requirement, not an afterthought — is the subject of this article.

Documents are chunked, embedded and indexed — vectors plus keywords. 1/4

What a RAG-Tuned Model Is

A RAG-tuned model is post-trained for the retrieval workflow itself. Three behaviors are part of the training, not the prompt: generating search queries from a conversation, synthesizing an answer across multiple retrieved documents while ignoring irrelevant ones, and emitting fine-grained citations alongside the generated text. A general-purpose model with a RAG prompt template approximates these behaviors. A RAG-tuned model is optimized and evaluated on them, which is a different engineering statement.

Command R targets what Cohere calls the scalable category: models that trade some peak capability for latency, throughput, and price. The underlying claim is that production RAG rarely needs the largest available model. It needs a model that reliably stays on-source, at a cost that survives contact with real query volumes.

Command R in Numbers

Command R is a 35-billion-parameter model with a 128k-token context window, priced at $0.50 per million input tokens and $1.50 per million output tokens. For comparison: GPT-4 Turbo (November 2023) costs $10.00 and $30.00 for the same volumes. Cohere's own benchmarks place Command R ahead of GPT-3.5 Turbo on end-to-end RAG tasks. Vendor benchmarks warrant skepticism; the pricing is a fact.

Two further numbers matter for European deployments. The model covers 10 business languages, including German, French, Spanish, Japanese, and Arabic — relevant wherever source documents are not in English. And the weights are public on Hugging Face under a CC-BY-NC license: open for research and evaluation, explicitly not for commercial self-hosting.

Property	Command R (March 2024)
Release date	11 March 2024
Parameters	35 billion
Context window	128k tokens
API pricing	$0.50 / $1.50 per 1M input/output tokens
Languages	10 business languages including German
Weights	Open on Hugging Face (CC-BY-NC / non-commercial)

Citations as an API Contract

The mechanism is an API contract. Cohere's Chat endpoint accepts a documents parameter: a list of text snippets with IDs, typically the output of your retrieval pipeline. The response contains, next to the generated text, a list of citation objects. Each carries start and end character indices into the generated answer, the cited span, and the IDs of the supporting documents. Citations are structured data, not prose.

This changes what a product can do. A UI can highlight exactly which sentence rests on which source. A test suite can compute grounding coverage — the share of answer text backed by citations — as a regression metric. A compliance layer can reject answers whose critical claims cite nothing. None of this is feasible when attribution lives somewhere in free text.

Tool Use as a Second Pillar

Command R launched with a second trained capability: tool use. The model emits structured function calls — to search endpoints, databases, calculators, internal APIs — and incorporates the results into a grounded answer. Retrieval itself becomes one tool among several. That unifies RAG and system integration under a single mechanism, and it is why we treat RAG-tuned and tool-tuned as one category, not two.

Scoping matters here. As of today, the API supports single-step tool use: one round of calls, then an answer. Chaining — using one tool's output to decide the next call — remains application code, typically a hand-written loop or an orchestration framework, with the latency and debugging cost that implies. Models trained natively to plan multi-step tool sequences are the obvious next step. They do not exist in Cohere's lineup today.

What Grounded Generation Does Not Solve

Grounded generation attributes; it does not verify. If the retrieved document is wrong or outdated, the model produces a confidently cited wrong answer. Citations move the trust question from the model to the corpus. That is an improvement — a corpus can be curated, versioned, and audited — but it is not a solution. Garbage in remains garbage out, now with page references.

Retrieval quality stays decisive. A citation-capable generator cannot repair a search step that returns the wrong passages, which keeps embedding models such as Cohere's Embed v3 (November 2023), rerankers, and chunking strategy on the critical path. The 128k window does not remove retrieval either: cost and latency scale with input tokens, and filling the context is the expensive way to be lazy. Grounding reduces hallucination. No vendor claims it eliminates it, and none should.

Outlook From April 2024

Our expectations, stated on 1 April 2024. First, the scalable category fills up fast: Command R will not remain alone in its class, and a larger sibling with the same RAG and citation training is a predictable release within months, not years. Second, multi-step tool use moves from application frameworks into the models themselves in the course of 2024 — the training recipe that produces citations can produce plans.

Third, the larger bet: by 2025, machine-readable citations will be a standard line item in enterprise RAG procurement, the way SSO is for SaaS. At Blue IT Systems we already treat citation spans as part of the system interface — tested, versioned, monitored — not as UI decoration. Grounded generation as a product requirement rather than an afterthought is the bar Command R has set. We do not expect the market to lower it.