Khalid ShamsPrincipal Solutions Architect

Learn AI The SHAMS AI Atlas How a model is made

How a model is made, before anyone calls it.

Second lesson. Follow one language model from raw text to the pinned version the flagship scene calls. Teaching simulation · synthetic data

How a model is madeA teaching drawing in three columns. On the left, nested rings place large language models inside deep learning, machine learning and AI, above shelves of text, a chart of who is represented and a held-out slice. In the middle, a tokenizer cuts text into tokens, an embedding map turns them into numbers, a stack of transformer blocks with attention and small networks scores the next token, parameter dials and one enlarged unit sit beside it, and a training loop predicts, compares and adjusts, with a loss chart. On the right, the saved base model goes through fine-tuning, preference training with a person choosing between answers, an evaluation bench and release as a pinned version. A text version of every step follows the drawing. FRAME AND DATAPRETRAINING · THE TRAINING LOOPAFTER PRETRAINING AIMACHINE LEARNINGDEEP LEARNINGLLMTaxonomyAI > ML > deep learning> large language models(nested, overlapping) TEXT CORPUS · UNLABELEDWHO IS REPRESENTEDHELD-OUT SLICEnever trained onDataset storesources and licencesfilters, dedupowners per sourcerepresentation profileheld-out split(evaluation only) TOKENIZERThe lamp arrived crackedThelamparrivedcracked6 tokens with this tokenizerTokenizertext to token idsfixed vocabulary EMBEDDINGScrackedbrokendamagedrefundorderlamp2 of many axesEmbedding tabletoken id to vectorlearned in traininghundreds of dimensions TRANSFORMERBLOCK 1BLOCK 2BLOCK 3BLOCK N· · ·×NSCORES FOR THE NEXT TOKENcracked .12 · broken .09TransformerN x (attention + MLP)residual streamoutput: probabilityfor every token inthe vocabulary PARAMETERS8 drawn · billions realWeightstensors set bythe optimizer;fixed after release ONE UNIT, ENLARGED×0.9×0.4×−0.3Σfoutput 0.43Unity = f(w · x + b) TRAINING LOOPPREDICT'arrived ___'COMPAREloss = 2.12ADJUSTbackpropagateagainRECIPE v3 · predict, compare, adjustTraining jobforward pass, loss, backward pass,optimizer step; checkpoints savedon a schedule LOSSsolid: trainingdashed: held outMetricstrain lossheld-out loss CHECKPOINTbase-v0architecture + learned parametersCheckpointweights + config FINE-TUNING · WORKED EXAMPLES'Lamp arrived cracked'asks order no. + photos'Where is my refund?'checks status, no promiseSFT jobprompt and response pairsfrom base-v0 to sft-v1 PREFERENCE TRAININGAB✓raterPRINCIPLESB preferred; the model isshifted toward answers like BPreference trainingranked pairs to preference modelRL toward preferred outputs(human or AI feedback) EVALUATION · BEFORE RELEASEheld-out tasksrequests it should declinequality by groupEval suiteheld-out set, decline tests,per-group metrics; gates release RELEASEassistant-v1 · pinnedthe model card in the flagship scenerelease ownerModel registrypinned version served by the APIrollback to the previous version TRAINING LOG · SYNTHETICRun logtyped events for this teaching run (synthetic) textvectorspredictionadjust every dialskippedback to the data

Ready

Press "Follow the model" to start.

One model, followed from raw text to a released version: what it learns from, what training actually adjusts, what fine-tuning and alignment add, and what evaluation has to show before release.

Also in the atlas: Follow one request through a governed AI system Content version 2026.10.3 · reviewed against primary sources on 2 October 2026

Text version

The same lesson, in words.

Teaching simulation. The stages are real; the scale, numbers, names and examples are illustrative and far smaller than any production model. No part of this page trains or calls a model.

  1. Frame. Where this fits. The model in the flagship scene is one kind of AI: inside machine learning, inside deep learning, a large language model. This lesson follows how such a model is made, long before any application calls it.
  2. Data. Collect and filter text. A large body of text is gathered, cleaned and deduplicated. A held-out slice is set aside and never trained on. The bars show who and what is represented; gaps here become weak spots later.
  3. Tokens. Cut the text into tokens. A tokenizer splits text into tokens: whole words, parts of words, characters. Whether 'cracked' is one token or two depends on the tokenizer.
  4. Embed. Turn each token into numbers. Each token is looked up in a learned embedding table and becomes a vector, a long list of numbers. During training, tokens used in similar ways drift near each other.
  5. Architecture. Up through the stack of blocks. The vectors pass up through many identical blocks. In each, attention lets every token weigh the others, then a small neural network transforms each one. The top of the stack scores every possible next token.
  6. Architecture. Inside one unit. Zoom into one unit of that network: multiply each input by a weight, add them up, apply a simple function. The weights are parameters. At the start of training they are random.
  7. Train. Guess the next token. Training shows real text with the next token hidden: 'The lamp arrived ___'. The model gives every candidate a probability, for example cracked 12%, broken 9%, late 4%. The numbers here are illustrative.
  8. Train. Measure how wrong it was. The text said 'cracked'. The loss is one number for how little probability the model gave the right token. The training algorithm's whole aim is to make this number smaller.
  9. Train. Adjust every parameter a little. Backpropagation works out, for every parameter, which direction would have lowered the loss. Each one moves a tiny step that way. No person edits a rule; the numbers shift.
  10. Train. Repeat across the corpus. The loop runs an enormous number of times. Training loss falls, and loss on the held-out slice falls with it: the patterns carry over to text the model has not seen.
  11. Checkpoint. Save the base model. The architecture and its learned parameters are saved as a checkpoint, base-v0. It continues text fluently, but a pretrained model is not inherently good at answering questions or following instructions.
  12. Fine-tune. Fine-tune on worked examples. The base model is trained further on request and good-answer pairs written by people. It starts to answer like an assistant, and to copy any habits in those examples.
  13. Align. Train toward preferred answers. People, or a model guided by written principles, compare pairs of answers. A preference model learns those choices and reinforcement learning shifts the model toward the preferred ones. Wanted behaviour becomes more likely. It is not guaranteed.
  14. Evaluate. Test on what it never trained on. The candidate is tested on held-out tasks, on requests it should decline, and by group to look for uneven quality. Every release criterion is met.
  15. Release. Pin and release a version. The passing checkpoint is released under a fixed version name, and applications pin it: this is the model card in the flagship scene. Its parameters no longer change. Each call gets a fresh context.

How a run can end

Release is a decision made on evidence. These are the ways this build can end.

The 22 concepts in this scene

Artificial intelligence Models
Software that works out from its input how to produce predictions, content, recommendations or decisions. A broad term. Rule-based systems, machine learning, deep learning and large language models all sit inside it. What makes a system 'AI' in current definitions is that it infers its outputs rather than following a fixed list of instructions for every case. Where the analogy stops: Rings suggest neat borders. In practice the terms overlap and are used loosely, often for marketing.
Data Data
The examples a model learns from. For a language model, a large collection of text, filtered and deduplicated. What is in it, how much, and what is missing all shape what the model can do and how it fails. Where the analogy stops: Shelves look neutral. Text carries the habits, gaps and biases of whoever wrote it.
Tokens Models
The small pieces of text a model reads and writes: words, parts of words, characters or bytes. Text is encoded into tokens before the model processes it, and a response is produced token by token. For Claude a token is about 3.5 English characters on average, varying by language. Context windows are measured in tokens. Where the analogy stops: Building bricks have fixed shapes. Tokenizers differ between models, so the same text can be a different number of tokens.
Embeddings Data
Lists of numbers that stand for pieces of text, placed so similar meanings sit close together. Inside a language model each token is turned into a vector by a learned embedding. Separately, embedding models produce vectors for whole passages, which retrieval uses to find similar text. Where the analogy stops: A flat map has two directions. Real embeddings have hundreds or thousands, and 'close' depends on the model.
Transformer Models
The model design behind today's large language models: blocks of attention and small networks, stacked. Tokens enter as embeddings; each block applies attention, then a small feed-forward network; many blocks are stacked; the last layer scores every possible next token. Introduced in 2017 and built entirely around attention. Where the analogy stops: A stack drawing looks like a pipeline of separate stages. Every block works on all tokens at once.
Parameters Models
The adjustable numbers inside a model. Training sets them; then they stay fixed. Every connection in the network has a weight, a number that scales what passes through it. Large language models have billions of them. Together they hold everything the model learned. Where the analogy stops: Dials suggest each one means something. Individual parameters rarely do; meaning is spread across many.
Artificial neuron Models
One small unit: multiply each input by a weight, add them up, apply a simple function. Inputs arrive, each is scaled by its weight, the results are summed with a bias term, and a simple function (often one that cuts off negative values) decides the output. Networks are built from many of these. Where the analogy stops: Loosely inspired by brain cells. An artificial neuron is arithmetic and nothing more.
Training Models
Adjusting a model's parameters, example by example, until its outputs fit the data. For a language model: show text, ask the model to predict the next token, measure how wrong it was (the loss), adjust every parameter slightly to be less wrong, and repeat over a huge amount of text. This first stage is pretraining. Where the analogy stops: Practice makes a person better at many things at once. Training improves exactly what the loss measures, and nothing else is promised.
Model (the trained file) Models
What training produces: the architecture plus its learned parameters, saved as a versioned file. After pretraining, the base model continues text well but is not inherently good at answering questions or following instructions. Later stages start from this checkpoint and produce new ones. Where the analogy stops: A saved file sounds finished. A base model is a starting point; fine-tuning and preference training shape how it behaves.
Fine-tuning Models
Further training a pretrained model on a smaller, focused set of examples. The model's parameters are adjusted again, this time on examples of the behaviour wanted, such as answering in an assistant's manner or a domain's style. It starts to mimic that dataset, including its flaws. Where the analogy stops: Coaching a person adds a skill. Fine-tuning can also shift behaviour you did not intend to touch.
Alignment Models
Training a model toward the behaviour people want and away from what they do not. Raters (or another model guided by written principles) compare pairs of responses; a preference model learns those judgments; reinforcement learning then pushes the model toward preferred responses. This makes wanted behaviour much more likely. It does not guarantee it. Where the analogy stops: A person who agrees to rules can be held to them. A model's training shifts likelihoods; it signs nothing.
Goal verification Control
Checking the work against the goal before calling it done. The run succeeds when the application verifies the success criteria (refund recorded, amount within policy, customer told), not when the model says it has finished. Where the analogy stops: A tick looks final; verification is only as good as the criteria.
Model (LLM) Models
The engine that reads what is on the desk and writes a response. An autoregressive language model produces output one token at a time (tokens are word pieces). It can propose a tool call, but it does not execute anything itself. Where the analogy stops: An engine suggests mechanism; a model is learned statistics, and the same input can produce different output.
Event trail Control
A trail of stamped events you can follow back. Recorded events: model calls with stop reasons, proposed and executed tool calls, authorization decisions, approvals, errors and the final outcome. Where the analogy stops: The trail shows what was recorded, not why the model produced what it did.
Machine learning Models
Getting behaviour from examples instead of writing every rule by hand. A training procedure adjusts a model's parameters so its outputs fit the examples it is shown. The rules end up in the numbers, learned from data, not written by a person. Where the analogy stops: 'Learning' suggests understanding. The model fits patterns in its training data, including patterns nobody intended.
Deep learning Models
Machine learning with many stacked layers, each building on the one before. Each layer turns the previous layer's output into a slightly more useful representation. Stacking many layers lets the model learn complex patterns, such as grammar or meaning, from raw input. Where the analogy stops: 'Deep' means many layers, not deep understanding.
Bias Data
Systematic unfairness or error that comes in with the data, the design or the people involved. If some groups, dialects or situations are rare in the data, the model will usually do worse for them. Bias can also come from how labels were made and from human choices across the whole process. Where the analogy stops: A bar chart makes it look like one number. Bias has many sources and shows up in different ways for different uses.
Attention Models
The step where each token weighs the other tokens in its context to decide what matters for it. For every token, attention computes how relevant each other token is and mixes in information from the relevant ones. It is how 'it' gets linked to 'the lamp' many words earlier. Where the analogy stops: Human attention is a spotlight with intent. Model attention is a computed weighting with no intent.
Neural network Models
Layers of artificial neurons connected by weighted links. Each layer's outputs become the next layer's inputs. With the right weights the whole network turns an input (tokens) into a useful output (a prediction of the next token). Where the analogy stops: A wiring diagram suggests you can trace why an answer came out. With billions of weights, you mostly cannot.
Backpropagation Models
Working out how much each parameter contributed to an error, so each can be nudged the right way. After the loss is measured, the error is passed backwards through the layers to compute, for every parameter, which direction would have reduced it. The optimizer then moves each parameter a small step that way. Where the analogy stops: 'Learning from mistakes' suggests reflection. It is calculus applied to one number, the loss.
Algorithm Models
A precise, step-by-step procedure a computer follows. Training is itself an algorithm: show examples, measure the error, adjust the parameters, repeat. The algorithm is written by people; the parameters it produces are learned. Where the analogy stops: A recipe gives the same dish every time. Training runs can differ with data order and random starting values.
Supervised learning Models
Learning from examples that come with the right answer attached. Each example pairs an input with a label or a good output, and the model is adjusted toward that answer. Fine-tuning a language model on prompt and good-answer pairs is a form of it. Where the analogy stops: A teacher can explain why an answer is right. A label only says what the answer is.

Sources

The workshop metaphor, the scenario and its numbers are teaching devices written for this site. They are not quoted from the sources.