2G

AI jargon spreads faster than understanding. This guide is for engineers who keep hearing words like tokens, RAG, context window, and 7B model and want explanations they will actually remember.

If you have spent any time around modern AI tools, you have probably heard people say things like:

• “That model has a larger context window.”
• “The latency is fine, but tokens per second is bad.”
• “We don’t need fine-tuning. RAG is enough.”
• “We need stronger guardrails.”
• “This benchmark harness says Model A beats Model B.”

The problem is not that these terms are complex. The problem is that they are often explained in circular jargon:

“Embeddings are vector representations used for semantic retrieval.”

That sentence is technically fine and emotionally useless.

This post does something different. It explains the most-used LLM terms in simple language, using examples designed to stick in your head. The target reader is an engineer, so the explanations stay honest. But they should also be memorable enough that the next time someone says “this model is hallucinating because the prompt blew the context window,” you immediately know what they mean.

Index

#	Term	What it covers
1	Model	The trained AI system itself
2	Prompt	The input you give the model
3	System Prompt	Hidden instructions set by the application builder
4	Token	The unit the model reads and writes in
5	Context Window	How much text the model can see at once
6	Parameters	The learned internal values from training
7	Inference	Running the model to generate a response
8	Tokens Per Second	How fast the model generates output
9	Temperature	How predictable or adventurous the output is
10	Hallucination	Confident output that is factually wrong
11	Fine-Tuning	Further training a model on a narrower dataset
12	Embedding	A numeric representation of meaning
13	Vector Database	Where embeddings are stored and searched
14	RAG	Retrieving documents to ground model answers
15	Guardrails	Controls that prevent unsafe or invalid behavior
16	Tool Use / Function Calling	The model invoking external functions or APIs
17	Agent	An AI system that plans and acts across multiple steps
18	MCP	A standard protocol for connecting models to tools
19	Eval Harness	A repeatable setup for testing AI behaviour

---

How to Read This Guide

Each term follows the same pattern:

• Simple meaning: the plain-English definition
• Relatable example: the part you are unlikely to forget
• What people usually mean: how the term is used in real conversations
• Common confusion: where people mix it up with something else

The Mental Model First

Before the glossary, here is the shortest useful mental model of an LLM:

A large language model is a machine that has seen a vast amount of text and learned statistical patterns about what words or word-pieces are likely to come next.

It does not “think” like a human. It predicts the next chunk of text repeatedly, very quickly, while being steered by your prompt, its training, its safety rules, and the amount of prior text it is allowed to see.

If that sentence makes sense, the rest of the jargon becomes much easier.

The Jargon Map

These terms fall into four buckets:

Bucket	Terms
What the model is	model, parameters
How you talk to it	prompt, system prompt, token, context window
How it responds	inference, tokens per second, temperature, hallucination
How teams improve or control it	fine-tuning, embeddings, vector database, RAG, guardrails, tool use, agents, MCP, eval harness

---

1. Model

Simple meaning

A model is the trained AI system itself: the thing that takes your input and generates an output.

Relatable example

Think of models like specialists you hire.

• A general physician handles most problems but refers you out for complex cases.
• A cardiologist costs more but is exactly right for heart issues.
• A pediatric specialist is wrong for your problem if you are not a child.

All are doctors. All trained in medicine. But they are built for different tasks, cost differently, and perform differently on the same question.

What people usually mean

When people say “try a different model,” they usually mean:

• use a different AI system entirely
• switch to one that is cheaper, faster, smarter, or safer for the task

Common confusion

A model is not the same thing as an app like ChatGPT, Claude, or Copilot. Those products are interfaces and systems built around one or more models.

---

2. Prompt

Simple meaning

A prompt is the input you give the model: instruction, question, context, examples, or all of the above.

Relatable example

Imagine telling a waiter: “food.”

Versus: “One masala dosa, crispy, with extra sambar, no chutney, please bring it before the others at the table.”

Both are orders. Only one eliminates ambiguity.

The difference in output quality is almost entirely explained by input quality – not the chef.

What people usually mean

When engineers say “the prompt needs work,” they usually mean:

• the instructions are vague
• important context is missing
• the output format is underspecified
• the model is being asked to do too many things at once

Common confusion

Prompts are not magical spells. A better prompt improves odds. It does not turn a weak model into a miracle worker.

---

3. System Prompt

Simple meaning

A system prompt is a set of instructions given to the model before the conversation starts. It is written by the application builder, not the end user, and the user usually never sees it.

Relatable example

Think of it like the briefing a manager gives a new employee before their first customer call:

“Be polite. Never discuss competitor pricing. Always end with ‘Is there anything else I can help you with?’ Do not process refunds without supervisor approval.”

The customer never hears that briefing. But every response the employee gives is shaped by it.

That is a system prompt: the invisible layer of instructions that frames how the model behaves before anyone types a single word.

What people usually mean

When engineers say “put it in the system prompt,” they mean bake the instructions into the application layer so the model follows them consistently across all users and sessions – without relying on users to ask correctly each time.

Common confusion

The system prompt is not a magic lock. Users can still influence model behaviour through their messages. In some cases, a malicious user can attempt to override system prompt instructions – this is called prompt injection, and it is a real security concern for production AI applications.

---

4. Token

Simple meaning

A token is a chunk of text the model reads or writes. It is smaller than a sentence and not always the same as a word.

Relatable example

Think of tokens like mobile data.

You do not buy “messages.” You buy megabytes. A short WhatsApp text barely touches your plan. A video drains it in seconds.

The model counts tokens the same way a carrier counts megabytes – usage, billing, and limits are all measured in that unit, not in the number of things you said.

What people usually mean

When people mention tokens, they are usually talking about:

• how much text fits into the model’s memory for one request
• how much the request costs
• how fast the response is generated

Common confusion

One token is not one word. Sometimes a short word is one token. Sometimes a longer word is split into multiple tokens. Code, punctuation, and JSON can tokenize in surprising ways.

---

5. Context Window

Simple meaning

A context window is the amount of text the model can consider at once in a single conversation or request.

Relatable example

Imagine you are working with a teammate in a room with only one whiteboard.

Everything relevant to the task has to fit on that whiteboard:

• your instructions
• the copied text from documents
• prior chat history
• intermediate notes

Once the whiteboard is full, something has to be erased.

If the key requirement was written in the top-left corner an hour ago and got wiped out to make room for new notes, the teammate stops using it. Not because they are careless, but because it is no longer on the board.

That is the context window.

What people usually mean

When people say “we need a larger context window,” they usually mean:

• the prompt plus documents plus chat history no longer fit well
• the model is forgetting earlier details
• the use case involves long docs, long code files, or many prior turns

Common confusion

A larger context window does not automatically mean better reasoning. It just means the model can see more text at once.

---

6. Parameters

Simple meaning

Parameters are the internal learned values inside the model. They are what the model adjusted during training to capture patterns in text.

Relatable example

Imagine a cricket player who has faced 200,000 deliveries over 15 years.

They cannot explain every micro-adjustment they make when a fast ball swings late. It is baked into muscle memory – posture, timing, weight shift – across billions of tiny corrections over time.

Parameters are that muscle memory. Not rules written down. Adjustments absorbed through repetition. When people say “7B parameters,” they mean seven billion of those tiny learned adjustments, all shaping every response.

What people usually mean

When people talk about parameter count, they are using it as a rough proxy for:

• model capacity
• possible intelligence or nuance
• infrastructure cost

Common confusion

Does more parameters mean more intelligence?

Not exactly – and this trips people up constantly.

More parameters means the model had more capacity to absorb patterns during training. Think of it like a bigger brain with more neurons. But neurons alone do not make someone smarter. What matters is what they were trained on, how well they were trained, and whether that training matches your problem.

A 70B model trained on poor data or misaligned tasks can lose to a well-trained 7B model on your specific use case. And a smaller model that runs 10x faster might be the right call even if the larger one scores slightly higher on benchmarks.

Parameter count is a rough proxy for capability, not a direct measure of intelligence or quality. When choosing a model, latency, cost, task fit, and eval results on your own data matter more than the parameter headline.

---

7. Inference

Simple meaning

Inference is the act of running the model to generate an answer after it has already been trained.

Relatable example

Training is like years of cooking school. Inference is dinner service at 8:15 PM when a customer orders mushroom risotto.

The learning happened earlier. Inference is the live act of using that learning to produce an answer right now.

What people usually mean

When engineers say “inference cost” or “inference latency,” they are talking about what happens when the model is actually used in production.

Common confusion

Inference is not training. Training changes the model. Inference uses the model as it already exists.

---

8. Tokens Per Second

Simple meaning

Tokens per second measures how fast the model generates output.

Relatable example

Imagine watching a live event where subtitles appear in real time.

• One system types out captions word by word as the speaker talks, matching the pace.
• Another pauses for 4 seconds, then drops the whole paragraph at once.

You eventually read the same text. But one experience feels alive. The other makes you feel like the system is struggling.

Tokens per second is what separates those two experiences. The final answer is the same. The perceived quality is not.

What people usually mean

When people talk about tokens per second, they are usually describing:

• perceived response speed
• whether the product feels snappy or sluggish
• whether streaming output feels pleasant

Common confusion

A model can have good total latency but still feel slow if generation drips out too slowly after it starts. Users notice that.

---

9. Temperature

Simple meaning

Temperature controls how predictable or adventurous the model’s output is.

Relatable example

Imagine asking three ad agencies for a tag line.

At low temperature, the copy stays safe and predictable:

• “Fast Delivery, Trusted Service”
• “Reliable Service for Everyday Needs”
• “Quality You Can Count On”

At high temperature, they get more adventurous:

• “Delivered Before You Even Start Worrying”
• “Your Deadline’s Worst Nightmare”
• “So Reliable It Feels Like Cheating”

That is temperature: how conservative or adventurous the output is.

What people usually mean

Low temperature is usually preferred for:

• factual answers
• code generation
• structured output

Higher temperature is more useful for:

• brainstorming
• creative writing
• divergent idea generation

Common confusion

Temperature does not make the model more intelligent. It changes how conservative or varied the output is.

---

10. Hallucination

Simple meaning

A hallucination is when the model states something false, made up, or unsupported as if it were true.

Relatable example

Imagine a new intern who was asked in a meeting: “What is the refund policy for orders over 90 days old?”

They do not know. But instead of saying that, they answer confidently:

“Yes, 90-day orders are eligible for a full refund, as long as the item is unopened.”

It sounds like a real policy. It was not. They filled the gap with something plausible.

That is a hallucination. Not malicious. Not random noise. A confident, well-formed answer to a question the model could not actually answer.

What people usually mean

When people say a model is hallucinating, they usually mean:

• it fabricated facts
• it cited nonexistent sources
• it confidently filled a gap with nonsense

Common confusion

Hallucination is not the same as lying. The model is not usually trying to deceive. It is producing plausible-looking text that is not grounded in reality.

---

11. Fine-Tuning

Simple meaning

Fine-tuning means further training a model on a narrower dataset so it behaves better for a specific domain or style.

Relatable example

Imagine a doctor who completes a general MBBS and then does a 3-year fellowship in radiology.

Same foundational training. Same general knowledge. But now they can read a CT scan with a level of nuance a generalist cannot match.

Fine-tuning does that for a model: same base, narrowed and sharpened for a specific domain.

What people usually mean

When teams discuss fine-tuning, they usually want one or more of these:

• a specific tone or style
• better behavior on domain-specific tasks
• stronger performance on repeated structured tasks

Common confusion

Fine-tuning is not the first answer to every AI problem. Many teams should first improve prompts, retrieval, and evaluation before reaching for it.

---

12. Embedding

Simple meaning

An embedding is a numeric representation of meaning. It turns text into coordinates in a mathematical space so similar meanings end up closer together.

Relatable example

Imagine a bookstore where books are not arranged alphabetically, but by “what feels similar.”

• books on startups end up near books on product building
• books on grief end up near books on healing
• books on gardening end up far from books on compiler design

An embedding is what lets software place text on that similarity map.

What people usually mean

When engineers mention embeddings, they are usually talking about:

• semantic search
• finding similar chunks of text
• retrieval systems

Common confusion

Embeddings do not directly generate answers. They help systems find relevant information.

---

13. Vector Database

Simple meaning

A vector database is a database built specifically to store and search embeddings – the numeric representations of meaning. It lets you find content by similarity rather than by exact keyword match.

Relatable example

A regular database is like a filing cabinet. You find things by their exact label: “Invoice_March_2024.pdf.”

A vector database is like a librarian with a very good memory for what things are about. You say “something about project delays in Q1” and they pull out everything relevant – even if none of those documents use the exact phrase “project delays in Q1.”

That is what a vector database enables: search by meaning, not just by keyword.

What people usually mean

In most engineering conversations, vector databases come up alongside RAG. The embeddings of your documents are stored in the vector database, and at query time the system retrieves the most semantically similar chunks to feed into the prompt.

Common examples: Pinecone, Weaviate, pgvector, Chroma.

Common confusion

A vector database does not replace your regular database. They solve different problems. You still need a relational or document database for structured data, transactions, and exact lookups. The vector database handles the “find things that mean something similar to this” problem.

---

14. RAG

Simple meaning

RAG stands for Retrieval-Augmented Generation. It means the model first retrieves relevant information from external sources, then uses that information to answer.

Relatable example

Without RAG, asking the model about your company policy is like asking a smart lawyer to answer from memory.

With RAG, it is like letting the lawyer quickly open the exact policy binder, flip to the right page, and answer with the document in front of them.

That is why RAG is powerful: it grounds the answer in retrieved material instead of pure memory.

What people usually mean

When teams say “we should use RAG,” they usually mean:

• the model needs access to fresh or private information
• answers should come from actual docs, not memory alone
• the system should ground responses in retrieved context

Common confusion

RAG is not “the model learned our documents.” Usually the model did not absorb them permanently. The system fetched the relevant docs at request time.

---

15. Guardrails

Simple meaning

Guardrails are the controls around an AI system that prevent unsafe, invalid, disallowed, or low-quality behaviour.

Relatable example

Imagine a streaming platform for children.

The platform still has a full content library- including adult, horror, and violence. None of that was deleted. The system just controls what gets served based on who is watching.

Guardrails work the same way around an AI system. The capability may still exist. The controls determine what gets through, in what context, to whom.

What people usually mean

Guardrails can include:

• input filters
• output validation
• policy checks
• tool-use restrictions
• JSON schema validation
• refusal logic for disallowed requests

Common confusion

Guardrails are not the same thing as the model itself. They are often system-level protections built around the model.

---

16. Tool Use / Function Calling

Simple meaning

Tool use (also called function calling) is the ability for a model to invoke external functions, APIs, or services as part of generating a response – instead of answering from memory alone.

Relatable example

Imagine asking a smart assistant: “What is the weather in Mumbai tomorrow?”

Without tool use, the assistant guesses from training data, which may be months out of date.

With tool use, the assistant says: “Let me check” – opens a weather API, reads the result, and then answers with live data.

The model does not know the weather from memory. It knows how to ask the right tool, read the answer, and fold it into a response. That is tool use.

What people usually mean

When engineers talk about function calling, they mean defining a set of functions the model can choose to invoke with structured arguments. The model decides when and which function to call. The application actually runs it and returns the result. The model reads that result and continues.

This is how AI systems look up databases, send emails, run code, query APIs, or take action in external systems.

Common confusion

The model does not directly execute code or call APIs. It generates a structured request: “call this function with these inputs.” The application handles the actual execution and returns the result. The model only orchestrates – it does not run anything itself.

---

17. Agent

Simple meaning

An agent is an AI system that does more than produce one response. It can plan steps, use tools, inspect results, and continue toward a goal.

Relatable example

A chatbot is like asking a receptionist one question.

An agent is like hiring an event manager and saying:

Find a venue, compare prices, call three caterers, check the guest count, and come back with a plan.

The difference is not just “more text.” The difference is multi-step action toward an outcome.

What people usually mean

When people say “we need an agent,” they usually mean:

• the system must perform multi-step work
• it may need tools, memory, or iteration
• one-shot prompting is not enough

Common confusion

Not every workflow needs an agent. Sometimes a simple prompt plus a good UI is better, cheaper, and more reliable.

---

18. MCP

Simple meaning

MCP stands for Model Context Protocol. It is an open standard for connecting AI models to external tools, data sources, and services – so that any compatible model can work with any compatible tool without custom integration code for each pair.

Relatable example

Think of MCP like USB-C.

Before USB-C, every device had its own charging port. Laptops, phones, and cameras each needed a different cable. Every new combination required a new adapter.

USB-C standardized the connection. One port, one cable, many devices.

MCP does the same for AI tools. Instead of building a custom connector between every model and every data source, MCP defines a standard interface. A model that speaks MCP can work with any tool that also speaks MCP – file systems, databases, browsers, GitHub, Slack – without reinventing the integration each time.

What people usually mean

When engineers mention MCP, they are usually talking about plugging standardized tools into an AI system and having the model use them reliably. It removes the plumbing work and lets teams focus on what the tools actually do rather than how they connect.

Common confusion

MCP is not a model or an AI product. It is a protocol – a specification for how things talk to each other. The intelligence still comes from the model. MCP just handles the wiring.

---

19. Eval Harness

Simple meaning

An eval harness is a repeatable setup for testing AI behavior against a set of prompts, tasks, or benchmarks.

Relatable example

Imagine assessing a new driver by sitting in the passenger seat once and saying: “Seemed fine.”

Now imagine a standardized driving test: same route, same assessment criteria, same scoring rubric, applied consistently to every candidate.

An eval harness is the second approach. Same prompts, same scoring, same conditions – repeated reliably. Without it, you are judging the model the first way: impressions rather than evidence.

What people usually mean

When teams mention an eval harness, they usually want:

• repeatable model comparison
• prompt regression testing
• confidence before changing models or prompts

Common confusion

An eval harness is not just a benchmark headline on a marketing page. It is an internal system for testing behaviour that matters to your own use case.

---

The Terms People Most Commonly Mix Up

Parameters vs context window

• Parameters are what the model learned during training
• Context window is how much text it can see right now

One is the muscle memory baked in over years of training. The other is the size of the whiteboard available for the current problem.

Fine-tuning vs RAG

• Fine-tuning changes the model’s behaviour
• RAG changes what information is supplied at runtime

One sends the doctor back for a fellowship. The other hands the lawyer the correct binder before they speak.

Prompt vs guardrails

• Prompt tells the model what you want
• Guardrails constrain what the system is allowed to do

One is the request. The other is the safety fence.

Chatbot vs agent

• A chatbot responds
• An agent takes multi-step action

One answers your question at the desk. The other goes off and runs the entire errand.

Prompt vs system prompt

• A prompt is what the user sends
• A system prompt is what the application builder sends before the user ever types anything

One is the customer’s order. The other is the briefing the manager gave before the shift started.

Tool use vs agent

• Tool use is a single capability: the model can call a function and use the result
• An agent is a system built around that capability: plan, call tools, observe results, repeat

Tool use is one move. An agent is a full strategy session.

Embedding vs vector database

• An embedding is the numeric representation of meaning
• A vector database is where those embeddings are stored and searched

The embedding is the coordinate. The vector database is the map that lets you find nearby coordinates quickly.

If You Remember Only 10 Ideas

1. An LLM works by predicting the next token repeatedly.
2. Tokens are the building blocks of text the model reads and writes.
3. Context window is the model’s short-term working memory for one request.
4. Parameters are the learned internal knobs from training. More parameters is not the same as more intelligence.
5. Hallucination is confident falsehood, not magical intelligence gone rogue.
6. RAG gives the model relevant documents at runtime. Fine-tuning changes the model itself.
7. The system prompt is what the application builder sets before the user ever types a word.
8. Tool use is how models go beyond generating text to take real action in the world.
9. A vector database finds content by meaning, not by keyword. It is what makes RAG fast at scale.
10. Good AI systems are not just models. They are models plus prompts, system prompts, retrieval, guardrails, tools, and evaluation.

Final Takeaway

Most AI jargon becomes much less intimidating once you sort it into a few buckets:

• what the model is
• how you talk to it
• how it responds
• how teams make it useful and safe

If you understand those buckets, the buzzwords start sounding less like magic and more like normal engineering trade-offs.

And that is the real goal of AI 101: not memorizing fashionable terms, but being able to hear them in a meeting and know what they imply about system design, cost, speed, quality, and risk.