Foundation Models & Interfaces
This domain accounts for 22% of the AI-FOUND exam — the single largest domain. It tests your ability to identify the right foundation model for a given requirement, understand how models are built and measured, and know how to interact with them through chat interfaces and APIs. Expect scenario-based questions that require you to distinguish between model families, context window limits, API roles, and capability tradeoffs. Memorize exact model names, parameter counts where cited, and architectural constraints.
What Is a Large Language Model (LLM)
A Large Language Model is a neural network trained on massive text corpora to predict the next token in a sequence. Understanding the three core building blocks is essential for the exam.
Training Data
LLMs learn statistical patterns from pre-training data — web crawls (Common Crawl), books, code repositories, and curated datasets. The quality, recency, and diversity of this data directly determine a model's knowledge cutoff and capability profile. After pre-training, models are refined using RLHF (Reinforcement Learning from Human Feedback) to align outputs with human preferences.
Parameters
Parameters are the numerical weights adjusted during training. They store the model's "knowledge." Parameter count is a rough proxy for capability:
- GPT-3: ~175 billion parameters
- GPT-4: not officially disclosed, estimated 1T+ (mixture of experts)
- Llama 3 70B: 70 billion parameters
- Phi-3 Mini: ~3.8 billion parameters
More parameters generally mean higher capability but also higher inference cost and latency.
Tokenization
Tokens are the units models read and produce — roughly 0.75 words per token in English (1 token ≈ 4 characters). Tokenization is model-specific. The context window is measured in tokens, not words. Key tokenization facts:
- Whitespace, punctuation, and rare words often become multiple tokens
- Code and non-English text tend to use more tokens per word
- Tokenizers like BPE (Byte Pair Encoding) and SentencePiece are common
| Concept | Definition | Exam Relevance |
|---|---|---|
| Pre-training | Self-supervised next-token prediction on large corpus | Sets knowledge cutoff |
| Fine-tuning | Supervised training on task-specific data | Adapts base model |
| RLHF | Human preference feedback shapes outputs | Makes models "assistant-like" |
| Token | Atomic unit of text for the model | Drives cost and context limits |
| Parameter | Learned weight in the neural network | Proxy for model size/capability |
Common traps:
- Confusing tokens with words — always convert (1,000 tokens ≈ 750 words)
- Assuming a larger parameter count always wins — smaller, newer models often outperform older large models on specific tasks
- Treating the training cutoff as the deployment date — models are released months after their data cutoff
ChatGPT and GPT-4o: OpenAI's Flagship Model
ChatGPT is OpenAI's consumer chat interface; GPT-4o ("omni") is the underlying model powering it as of 2024. The exam distinguishes between the interface and the model.
GPT-4o Key Facts
- Multimodal: natively handles text, image, and audio input and output in a single model
- Context window: 128,000 tokens
- Training cutoff: April 2023 (knowledge cutoff for base model)
- Speed: significantly faster than GPT-4 Turbo; positioned for real-time use
- GPT-4o mini: smaller, cheaper variant for lower-stakes tasks
Model Tier Comparison
| Model | Context Window | Modalities | Relative Cost | Best For |
|---|---|---|---|---|
| GPT-3.5 Turbo | 16,385 tokens | Text | Lowest | High-volume, simple tasks |
| GPT-4 Turbo | 128,000 tokens | Text + Vision | High | Complex reasoning, long docs |
| GPT-4o | 128,000 tokens | Text + Vision + Audio | Medium-High | Real-time multimodal |
| GPT-4o mini | 128,000 tokens | Text + Vision | Low | Cost-sensitive applications |
| o1 / o1-mini | 128,000 tokens | Text | High | Multi-step reasoning, math |
Capabilities and Limitations
Capabilities: Code generation, multi-step reasoning, summarization, translation, vision understanding, function calling, structured output (JSON mode).
Limitations: Knowledge cutoff (no real-time data without tools), hallucination, context window cap, no persistent memory by default.
Common traps:
- Confusing ChatGPT (interface) with GPT-4o (model) — questions may test whether you know the distinction
- Assuming o1 models are faster — they are slower due to internal chain-of-thought reasoning but more accurate on math/logic
- Treating GPT-4 and GPT-4o as identical — GPT-4o adds native audio and is faster/cheaper
Sign in and enroll free to read the full guide
11 more sections (about 2,585 words) continue below. This course is free: enroll from the course page to unlock every guide.