A Large Language Model never actually “knows” the next word. It calculates the odds on every possible word, every single time.
mcqquestion.com
Science & Technology · SciTech0136
Statistics dressed up as fluency.
₹10,000cr
IndiaAI Mission outlay
Backing Sarvam AI, Gnani.ai, and BharatGen’s sovereign LLMs, unveiled Feb 2026.
Sarvam AI’s largest
105B params
Built and trained fully in India
BharatGen’s Param2
17B params
All 22 scheduled Indian languages
BharatGen funding
₹900cr
Largest single IndiaAI beneficiary
What the PYQ tested
Next-word predictionCorrect
Trained via optimizationCorrect
Produces unbiased outputFalse
Core building blocks
TokensWords/word-parts
Self-attentionTransformer core
Gradient descentTraining method
🔤 How an LLM Actually Predicts a Word
1. Tokens and Probabilities
- Tokens An LLM breaks text into small chunks called tokens, roughly words or parts of words.
- Probability Distribution Given the text so far, it calculates a probability for every possible next token, based on patterns learned from its training data.
- Picking a Word It then picks a token, typically the highest-probability one, appends it, and repeats the process to build a full response.
2. The Transformer Architecture
- Attention Modern LLMs use a transformer architecture, built around a mechanism called self-attention.
- What Attention Does Self-attention lets the model weigh how strongly every earlier word in a passage relates to the word it is about to generate.
- No Recurrence Unlike older models, transformers process a whole passage at once, rather than word by word in strict sequence, which makes them far faster to train at scale.
⚙️ Training an LLM
1. Learning by Predicting
- Self-Supervision During training, an LLM is shown huge amounts of text and asked to predict each next word from the words before it.
- Prediction Error When it predicts wrong, that error is measured mathematically as a loss value.
- Optimization Algorithms like gradient descent then adjust the model’s billions of internal parameters, step by step, to reduce that loss.
2. Scale Is the Point
- Parameters “Large” in Large Language Model refers to this parameter count, often tens or hundreds of billions of adjustable values.
- Data Training data is drawn from massive text collections: books, articles, code, and web pages.
- Compute Training at this scale needs large clusters of specialised chips, mainly GPUs, running for weeks at a time.
Test Yourself
⚠️ Limits, Bias, and Hallucination
1. Why LLM Outputs Are Not Unbiased
- Data Reflects the World An LLM learns only from its training data. Whatever imbalances and stereotypes exist in that data can be absorbed by the model.
- No Fact-Checking Layer The model has no separate mechanism verifying truth or fairness. It just predicts statistically likely text.
- The Common Trap A statement claiming LLMs “produce unbiased outputs” is false. Bias from training data is a well-documented, active research problem.
2. Hallucination
- Definition Hallucination is when an LLM generates text that sounds fluent and confident but is factually wrong.
- Why It Happens Because the model is optimising for plausible-sounding word sequences, not verified facts, a confident wrong answer looks the same to it as a confident right one.
- Practical Impact This makes independent verification essential before relying on LLM output in medicine, law, or any high-stakes decision.
📰 Current Affairs
1. India’s Sovereign LLM Push
- IndiaAI Mission Under the ₹10,000 crore IndiaAI Mission, the government backed several Indian companies to build sovereign large language models.
- AI Summit 2026 At the India AI Impact Summit in Delhi in February 2026, three sovereign AI models were unveiled: from Sarvam AI, Gnani.ai, and BharatGen.
- Sarvam AI Sarvam AI presented two models, built and trained in India, at 30-billion and 105-billion parameters.
- BharatGen BharatGen, led by IIT Bombay, launched Param2, a 17-billion-parameter model trained across all 22 scheduled Indian languages.
- Why It Matters The push aims to serve India’s many languages and reduce reliance on foreign models trained mostly on English-heavy data.
Previous Year Question
Asked as: “Which of the following statements with regard to Large Language Models (LLMs) used in machine learning is/are correct?” (UPSC CSP 2026, GS Paper I, Q42).
Leave a Reply