The Market Shift — What's Actually Happening
In 2024–2025, something changed. Teams started shipping features in days that used to take months. Companies started running entire customer support pipelines, code review agents, and document processing systems with 1–2 engineers instead of 10.
The common thread: those engineers weren't smarter. They knew how to orchestrate AI. They knew which AI primitives to reach for — and critically, they knew where AI breaks in production.
┌─────────────────────────────────┐ ┌─────────────────────────────────┐
│ Software Engineer │ │ AI Engineer │
│ │ │ │
│ • Writes APIs │ │ • Writes APIs │
│ • Builds CRUD │ │ • Knows when to use LLMs │
│ • Manages databases │ │ • Builds RAG pipelines │
│ • Deploys services │ │ • Orchestrates AI agents │
│ │ │ • Exposes tools via MCP │
│ Output: 1x │ │ • Knows how AI fails │
│ Replaceability: High │ │ │
│ │ │ Output: 10x │
│ │ │ Replaceability: Low │
└─────────────────────────────────┘ └─────────────────────────────────┘
Skill 1 — RAG and Vector Databases
RAG (Retrieval-Augmented Generation) is the pattern that makes LLMs actually useful in production. Instead of relying on an LLM's baked-in training data (which is stale and hallucination-prone), you inject relevant, up-to-date context into every prompt at query time.
Why LLMs Need RAG
A raw LLM has two problems: its knowledge has a cutoff date, and it will confidently make things up (hallucinate) when it doesn't know something. RAG solves both by giving the model the actual documents it needs to answer from.
── INDEXING (offline, runs once) ─────────────────────────────────
Your Documents (PDFs, DBs, APIs)
│
▼
┌─────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ Chunker │───▶│ Embedding Model │───▶│ Vector Store │
│ (split docs │ │ (text → 1536-dim │ │ (Pinecone/pgvec │
│ into ~500 │ │ float vector) │ │ tor/Qdrant) │
│ token chunks│ └──────────────────┘ └─────────────────┘
└─────────────┘
── QUERYING (runtime, per request) ───────────────────────────────
User: "What is our refund policy?"
│
▼
Embed query → [0.23, -0.81, 0.44, ...]
│
▼
Vector Store: cosine similarity search
│
▼
Top-K most relevant chunks returned
│
▼
┌─────────────────────────────────────────────────┐
│ Prompt = System + Retrieved Chunks + User Query │
└──────────────────────┬──────────────────────────┘
│
▼
LLM generates answer
grounded in real docs
Vector Databases — How They Work
A vector database stores embeddings — high-dimensional float arrays that represent the semantic meaning of text. Similar meaning = vectors close together in space. The DB lets you run approximate nearest neighbor (ANN) search across millions of vectors in milliseconds.
# Example: indexing + querying with pgvector (Postgres extension)
-- Enable extension
CREATE EXTENSION vector;
-- Store chunks with their embeddings
CREATE TABLE documents (
id SERIAL PRIMARY KEY,
content TEXT,
embedding vector(1536) -- OpenAI text-embedding-3-small dim
);
-- Find 5 most relevant chunks for a query vector
SELECT content
FROM documents
ORDER BY embedding <-> $1 -- cosine distance operator
LIMIT 5;WHERE name = 'foo', you query by semantic similarity. If you know SQL or NoSQL, you already know the mental model.
Skill 2 — AI Agents and How They Fail in Production
An AI agent is a system where an LLM doesn't just answer a question — it plans a sequence of actions, executes tools, observes results, and iterates until the goal is achieved.
This sounds magical. In practice, agents fail in specific, predictable ways. Knowing these failure modes is what separates an AI engineer from someone who just followed a tutorial.
How an Agent Works
User goal: "Book the cheapest flight to Delhi tomorrow"
│
▼
┌───────────────────────────────────────┐
│ LLM (Brain) │
│ │
│ Thought: I need to search flights │
│ Action: search_flights( │
│ origin="BOM", dest="DEL", │
│ date="2025-03-14") │
└─────────────────┬─────────────────────┘
│ tool call
▼
┌─────────────┐
│ Tool Layer │ ← your actual APIs
│ (search API) │
└──────┬───────┘
│ result: [{flight: AI-101, price: ₹3200}, ...]
▼
┌───────────────────────────────────────┐
│ LLM (Brain) │
│ │
│ Observation: Found 5 flights │
│ Thought: Cheapest is AI-101 ₹3200 │
│ Action: book_flight(id="AI-101", │
│ passenger=user_profile) │
└───────────────────────────────────────┘
│
▼
Final Answer: "Booked AI-101 for ₹3200"
How Agents Fail in Production
The agent gets stuck in a loop — calling the same tool repeatedly, or oscillating between two states. Without a hard step limit, it burns tokens and money forever.
Fix: Always set a max_iterations limit. Log every step. Alert when an agent exceeds N steps for a task that normally takes 3.
The LLM calls a real tool with made-up parameters — a user ID that doesn't exist, a date in the wrong format, a field name that's slightly wrong. The tool throws an error, and the agent spirals.
Fix: Validate all tool inputs strictly. Return structured errors that tell the LLM exactly what went wrong. Use JSON Schema to define tool parameters.
Each tool call adds to the conversation history. Long-running agents accumulate thousands of tokens. Eventually you hit the context limit — the agent loses its early memory and starts making inconsistent decisions.
Fix: Summarize completed steps. Use a scratchpad that compresses history. Only keep the last N tool results in context.
Agent sends an email, deletes a record, charges a card — and was wrong. Unlike CRUD APIs, these can't be rolled back by just hitting Ctrl+Z.
Fix: Classify tools as read-only vs write/destructive. Require human-in-the-loop approval for destructive actions. Build a dry-run mode for testing.
Skill 3 — MCP Servers and Tool Calling
MCP (Model Context Protocol) is an open standard developed by Anthropic that defines how AI models communicate with external tools and data sources. Think of it as a universal API contract between LLMs and the services they need to call.
Before MCP, every AI product had its own bespoke tool-calling format. MCP standardizes this — the same server can work with Claude, Cursor, and any other MCP-compatible client.
┌──────────────────────────────────────────────────────────┐
│ MCP Client (AI app) │
│ (Claude Desktop, your custom AI agent) │
└───────────────────────────┬──────────────────────────────┘
│ MCP Protocol (JSON-RPC over stdio/SSE)
┌────────────┼────────────┐
│ │ │
┌──────▼───┐ ┌────▼─────┐ ┌──▼───────┐
│ MCP │ │ MCP │ │ MCP │
│ Server 1 │ │ Server 2 │ │ Server 3 │
│ (GitHub) │ │(Postgres)│ │(Your API)│
└──────────┘ └──────────┘ └──────────┘
Each server exposes:
• Tools — functions the LLM can call (read_file, run_query)
• Resources — data the LLM can read (file contents, DB rows)
• Prompts — reusable prompt templates
Building Your Own MCP Server
// Simple MCP server in TypeScript (official SDK)
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
const server = new Server(
{ name: "my-api-server", version: "1.0.0" },
{ capabilities: { tools: {} } }
);
// Register a tool the LLM can call
server.setRequestHandler(ListToolsRequestSchema, async () => ({
tools: [{
name: "get_user_orders",
description: "Fetch all orders for a given user ID",
inputSchema: {
type: "object",
properties: {
userId: { type: "string", description: "The user's UUID" }
},
required: ["userId"]
}
}]
}));
// Handle tool execution
server.setRequestHandler(CallToolRequestSchema, async (req) => {
if (req.params.name === "get_user_orders") {
const orders = await db.query(
"SELECT * FROM orders WHERE user_id = $1",
[req.params.arguments.userId]
);
return { content: [{ type: "text", text: JSON.stringify(orders) }] };
}
});
// Connect via stdio (works with Claude Desktop, Cursor, etc.)
const transport = new StdioServerTransport();
await server.connect(transport);Tool Calling vs MCP
Where to Start — By Your Current Stack
30-Day Roadmap
- The engineer who orchestrates AI is irreplaceable. The engineer who only writes APIs is at risk of being replaced by the one who does.
- RAG = give LLMs real, up-to-date context at query time. Vector databases store semantic embeddings. You query by meaning, not by exact match.
- AI Agents plan and execute multi-step tasks. They fail in production via infinite loops, tool hallucination, context overflow, and irreversible actions. Know all four.
- MCP is the standard protocol for connecting LLMs to your tools and data. Build an MCP server once — works with any MCP-compatible client.
- If you know backend development, you're already 70% of the way to being an AI engineer. You have the system design instincts, the infra knowledge, and the debugging skills. You just need the AI primitives on top.