26 Nov 2023 AI Engineering 5 min read

OpenAI DevDay: What the Assistants API Changes for Builders

On November 6, 2023, OpenAI announced GPT-4 Turbo with a 128k context window, the Assistants API with Retrieval and Code Interpreter, and GPTs. We summarize the verified numbers, the new API primitives, and the concrete limits — and state what remains your responsibility: evaluation, data governance, cost control, and vendor risk.

One Keynote Rearranges the Stack

Until this month, teams building assistants on the OpenAI API assembled the stack themselves: conversation state, document chunking, embeddings, vector search, tool dispatch. On November 6, 2023, OpenAI held its first developer conference, DevDay, in San Francisco and moved several of these layers into the platform. The announced numbers explain the ambition: over 2 million developers on the API, 92 percent of Fortune 500 companies building on the platform, roughly 100 million weekly ChatGPT users.

This article states what changed for builders — model, API surface, pricing — and what did not change: evaluation, data governance and cost control remain on your side of the interface. All statements reflect the state as of November 23, 2023.

A task arrives — the agent plans its next step. 1/4

GPT-4 Turbo in Numbers

GPT-4 Turbo is available as gpt-4-1106-preview. Context window: 128,000 tokens, roughly 300 pages of text. Knowledge cutoff: April 2023, up from September 2021. Input tokens cost $0.01 per 1,000, output tokens $0.03 — 3x and 2x cheaper than GPT-4. The model adds JSON mode for guaranteed-valid JSON output and a seed parameter for reproducible sampling. GPT-3.5 Turbo gets a new 16K variant at $0.001 input and $0.002 output. Tokens-per-minute limits doubled for established GPT-4 customers.

Two caveats. The model is a preview; OpenAI announces the production release for the coming weeks. And a 128k window is capacity, not comprehension: whether the model reliably uses information from the middle of a long prompt is an empirical question you must test per task.

The Assistants API

The Assistants API introduces three server-side objects: an assistant (instructions plus tools), a thread (persistent message history), and a run (one execution of an assistant on a thread). You append messages; the platform manages history and truncation against the context window. Runs are asynchronous: you create a run and poll its status until it completes or requires action. Conversation state, previously your database schema, becomes an API resource.

This is OpenAI's first explicit step toward agent-like systems: an assistant can decide during a run to call tools, execute code, or search documents. The API has been in beta and open to all developers since November 6. It shares its foundation with the GPTs product in ChatGPT.

Retrieval and Code Interpreter

Retrieval accepts uploaded files and handles chunking, embedding, and search internally. No self-hosted vector database, no embedding pipeline. The limits are concrete: at most 20 files per assistant, at most 512 MB per file. Pricing is $0.20 per GB per assistant per day. Code Interpreter runs Python in a sandbox at $0.03 per session; a session stays active for one hour within a thread. Function calling completes the tool set.

What Retrieval does not do: it exposes no control over chunk size, embedding model, or ranking, and it reports no relevance scores. A corpus larger than 20 files does not fit. Code Interpreter's sandbox has no network access and processes only attached files. For large or quality-critical retrieval, a self-built RAG pipeline remains the correct choice.

GPTs Without Code

GPTs are configurable versions of ChatGPT: instructions, uploaded knowledge, and actions (HTTP calls to external APIs), assembled in a conversational builder without code. They can be shared by link; a GPT Store is announced for late November. Enterprise customers can restrict GPTs to internal use. GPTs are a ChatGPT product, not an API product — but they demonstrate the same primitives the Assistants API exposes to developers.

For engineering teams, GPTs matter in two ways. As a prototyping surface: an idea can be validated in an afternoon before any code is written. And as competition: if a configured GPT covers a use case, a custom application must justify itself through data access, integration, or control that ChatGPT cannot offer.

What Stays Your Responsibility

Evaluation stays with you. Retrieval is a black box; whether answers are actually grounded in your documents must be measured against your own gold set — before rollout and after every model or platform change. gpt-4-1106-preview is three weeks old; benchmark it against GPT-4 on your tasks, not on public leaderboards.

Data governance stays with you. Files and threads are stored on OpenAI's servers; for European customers this means DPA review, a retention policy, and a clear decision about which data may leave your infrastructure. Prompt injection through retrieved documents and tool outputs remains an open problem the platform does not solve.

Cost control stays with you. A single run can trigger several model calls over a growing thread; token consumption is not precisely predictable in advance. The API is in beta and carries no SLA. Budget guards belong in your integration layer, not in the backlog.

Pricing at a Glance

The table summarizes the prices announced on November 6 (per 1,000 tokens where applicable). Note the asymmetry: GPT-4 Turbo output remains three times more expensive than input, so prompt design that produces short, structured answers now has a direct price effect. Retrieval cost scales with stored gigabytes per assistant and day — an idle assistant with large files costs money daily.

Item	Price
GPT-4 Turbo input	$0.01 / 1K tokens
GPT-4 Turbo output	$0.03 / 1K tokens
GPT-3.5 Turbo 16K input	$0.001 / 1K tokens
GPT-3.5 Turbo 16K output	$0.002 / 1K tokens
Code Interpreter	$0.03 per session
Retrieval	$0.20 per GB per assistant per day

Outlook From November 2023

Our expectation from today's vantage point: the primitives of the Assistants API — persistent threads, runs, built-in tools — will become the reference model that other providers copy. Simple RAG chatbots will commoditize; differentiation moves to evaluation, domain data, and integration depth. We also expect the 20-file limit and the retrieval black box to loosen within months — developer pressure is already visible in OpenAI's own forums.

One more lesson from this November: between the 17th and the 21st, OpenAI's board removed Sam Altman and then agreed to his return. The platform is impressive; its governance proved volatile. We at Blue IT Systems therefore treat the Assistants API as one interchangeable backend behind our own abstraction layer — a general pattern we recommend to every team building on a single vendor.