Engineering notes
Since 2023 we have documented what actually works in AI systems — with numbers, sources and honest limits.
One year after Simon Willison named the vulnerability class, prompt injection remains unsolved. We define direct and indirect injection, explain why retrieval-augmented generation and tool use widen the attack surface, review the mitigations available in September 2023, and state plainly what each one does not do. No complete fix exists.
Chunking determines what a document QA system can retrieve at all. We compare fixed-size and recursive splitting, quantify chunk size and overlap trade-offs against the context windows of 2023, and explain why metadata is mandatory. With verified numbers from Pinecone, OpenAI and the Lost in the Middle paper — plus an outlook from September 2023.
Vector databases raised over 160 million dollars in spring 2023 alone. We explain what approximate nearest neighbor search actually does, how the HNSW index works, and why Postgres with the newly released pgvector 0.5.0 is often sufficient — plus the concrete criteria that justify a dedicated system.
Meta released Llama 2 on July 18, 2023 under a license that permits commercial use. We examine what open weights change for enterprise architectures: data residency without third-country transfers, fixed-cost inference instead of per-token billing, the measured quality gap to GPT-4, and where the 7B, 13B, and 70B models fit today.
Retrieval-augmented generation or fine-tuning: which puts company knowledge into a language model? A mid-2023 engineering view. We explain what fine-tuning actually changes, how embeddings and vector search work, why retrieval should be the default for facts, and where fine-tuning earns its cost — grounded in Lewis et al. (2020) and current tooling.
OpenAI added function calling to GPT-4 and GPT-3.5 on 13 June 2023. We explain what the new functions parameter enables — tool use, structured extraction, first agent loops — how the call cycle works, and where it fails: hallucinated arguments, unenforced schemas, prompt injection. With validation patterns that hold up in production.