Foundation Models: What They Are and How They Work
Foundation models are the underlying technology behind many of today's generative AI applications. If you're preparing for the AWS Certified AI Practitioner exam, you don't need to become a machine learning engineer to understand them. You do need to understand what makes a foundation model different from a traditional machine learning model, how these models are trained and used, and why the choice of model matters.
The easiest way to understand a foundation model is to start with its purpose.
A traditional machine learning model is often built to solve a relatively specific problem. You might train a model to determine whether an email is spam, predict the price of a house, or classify an image. The model is trained for that particular task.
A foundation model starts from a much broader base.
It is trained on a large and diverse collection of data so that it can learn general patterns and capabilities that can later be used for many different tasks. A language foundation model, for example, can be used for summarization, question answering, classification, content generation, translation, and other language-based tasks without requiring a separate model to be trained from scratch for each one.
That distinction is central to understanding generative AI.
Why this matters for AIF-C01: The exam is not simply testing whether you recognize the phrase foundation model. You may be given a business scenario and asked to identify the appropriate approach, service, or model. Understanding what makes a foundation model different from a traditional task-specific machine learning model gives you the context needed to reason through those questions rather than relying on memorized definitions.
What Makes a Model a Foundation Model?
The term foundation model refers to a model that has been trained on broad data at significant scale and can serve as a starting point for many downstream applications.
The important idea is reuse.
Instead of building a new model from the ground up for every application, an organization can start with an existing foundation model and adapt how it is used.
That adaptation might be as simple as providing a carefully written prompt. It might involve supplying additional information through retrieval. In some cases, the model itself can be customized or fine-tuned.
The foundation provides the starting capability. The application builds on top of it.
This is one reason generative AI has become so accessible. An organization does not necessarily need to collect an enormous dataset and train a large model from scratch simply because it wants an application that can generate or understand text.
AIF-C01 can test this distinction indirectly. A question might describe an organization that wants to build several different generative AI applications rather than a single narrowly defined prediction system. The important clue is the breadth of the intended use, not necessarily the terminology used in the question.
Foundation Model vs. Traditional Machine Learning Model
This distinction is worth understanding rather than memorizing.
Imagine a company wants to identify fraudulent credit card transactions.
A traditional machine learning approach might involve collecting transaction data, labeling transactions as fraudulent or legitimate, selecting features, training a model, and evaluating how well that model identifies fraud.
The resulting model has a specific job.
Now consider a large language foundation model. It may have been trained on a broad collection of text and learned patterns about language, relationships between words, and other characteristics of the data it was trained on.
That model isn't inherently a "customer service model" or a "summarization model." The same underlying model can support many applications.
This gives foundation models a different role:
| Traditional ML model | Foundation model |
|---|---|
| Often developed for a specific task | Designed to support many downstream tasks |
| Usually trained for a particular objective | Pre-trained on broad, large-scale datasets |
| New tasks may require another model | One model can support many applications |
| Training can require task-specific data | Existing capabilities can be adapted for new uses |
| Often produces predictions or classifications | Can support generation, understanding, reasoning, and other capabilities depending on the model |
The distinction isn't absolute. Traditional models can be highly capable, and foundation models can be specialized. The important point is that foundation models are intended to provide a reusable base for many applications.
AIF-C01 questions may give you several technically plausible choices. If the scenario describes a narrowly defined prediction or classification problem, don't automatically choose a generative AI foundation model simply because the question mentions AI. Start with the actual business requirement.
Pre-Training: Where the Foundation Comes From
The broad capabilities of a foundation model come primarily from pre-training.
During pre-training, the model processes a very large amount of data and learns patterns within that data.
For a language model, this can involve enormous amounts of text. The model doesn't simply store a collection of documents and retrieve them word-for-word. Instead, the training process adjusts the model's parameters so that it learns statistical patterns and relationships in the training data.
The result is a model with general capabilities that can later be used for specific tasks.
This is why the term foundation makes sense.
The model has already gone through a substantial training process before a particular organization decides to use it.
The organization can then build an application on top of that foundation rather than starting with an empty model.
For exam purposes, don't equate pre-training with every form of model customization. AIF-C01 covers several ways a model can be adapted or supplied with information, and those approaches solve different problems. If a question describes the initial large-scale learning process that establishes a model's general capabilities, you're looking at pre-training. If it describes supplying information at the time of a request, you're dealing with a different part of the architecture.
Parameters: What the Model Learns
A foundation model contains a large number of parameters.
Parameters are values that are adjusted during training as the model learns patterns from its training data. They are part of what allows the trained model to produce its outputs.
You will often see models described by the number of parameters they contain. For example, a model might be described as having billions of parameters.
More parameters does not automatically mean a model is better for every situation.
A larger model may have greater capabilities, but it may also require more computational resources and potentially have higher cost or latency. A smaller model may be preferable when an application has strict requirements around speed or cost.
For the AI Practitioner exam, the useful takeaway is not to memorize parameter counts.
Remember the relationship:
Training adjusts parameters → the trained parameters encode learned patterns → inference uses those parameters to produce outputs.
You generally don't need to memorize the parameter count of individual models for AIF-C01. What matters is understanding the tradeoff. If a scenario emphasizes model capability but also gives you requirements around cost, latency, or computational resources, don't assume that the model with the most parameters is automatically the correct choice.
Training vs. Inference
One of the most important distinctions in AI terminology is the difference between training and inference.
Training
Training is the process through which the model learns from data.
During training, the model's parameters are adjusted based on the training process.
Inference
Inference occurs when the trained model is used to produce an output.
When you send a prompt to a foundation model and receive a response, the model is performing inference.
A simple way to remember the difference:
Training creates or changes the model. Inference uses the model.
This distinction matters because generative AI applications spend a great deal of their time performing inference. Every time an application sends a request to a model and receives generated text, an inference operation has taken place.
This is a particularly useful distinction when reading AIF-C01 scenarios. A question may describe a user submitting a prompt to an already-trained model and ask what operation is taking place. That's inference, even though the model is actively processing information and producing a sophisticated response.
Training and inference are fundamentally different stages.
Tokens: The Pieces a Language Model Processes
Language models don't process text in exactly the same way humans read it.
Text is divided into smaller units called tokens.
A token might represent an entire word, part of a word, punctuation, or another piece of text. The exact way text is divided depends on the model and its tokenizer.
Consider:
"Foundation models are powerful."
The model doesn't necessarily treat each word as one token. Some words may be represented as multiple tokens, while common words may correspond to a single token.
Tokens matter for several reasons.
Context limits
Models have limits on how much information they can process within a particular context. These limits are generally expressed in tokens.
Cost
For services that charge based on tokens processed, the number of input and output tokens affects cost.
Performance
Processing more tokens generally requires more computation.
For the exam, remember that tokens are the units of text processed by language models, and token limits are closely related to context windows.
Token-related questions may connect to practical considerations rather than simply asking for a definition. If a scenario mentions input size, output size, context limits, or usage-based costs for a language model, tokens are likely part of the underlying issue.
Context Windows: What the Model Can Consider
A model's context window is the amount of information it can consider as part of a particular interaction.
This can include the user's prompt, previous conversation, instructions, retrieved information, and other content supplied to the model.
A larger context window allows an application to provide more information in a single request.
This becomes particularly important in applications that work with long documents or large amounts of retrieved information.
But there is an important distinction:
A context window is not the same thing as the model's training data.
If you provide a document to a model as part of a request, that document can become part of the current context without becoming part of the model's trained parameters.
That distinction becomes especially important when you start learning about Retrieval-Augmented Generation (RAG).
This distinction can become important when AIF-C01 presents a scenario involving large documents or a long conversation. A model's context window determines how much information can be considered within the interaction; it does not mean the model has permanently learned everything placed into that context.
Inference Does Not Mean Retraining
This is a common source of confusion.
Suppose you send a foundation model your company's internal policy document and ask:
"What is our company's policy on password rotation?"
The model can use the supplied document as context when generating its answer.
That does not mean the model has been retrained on the document.
The model's underlying parameters have not necessarily changed.
This distinction is one of the reasons techniques such as prompt engineering and RAG are so useful. They allow an application to provide information or instructions to a model without necessarily changing the underlying model itself.
This is an important AIF-C01 connection. A scenario may say that a company wants an AI application to answer questions using its frequently changing internal documents. The key requirement is not "teach the model everything about the company." The requirement is to provide relevant external information when the model generates an answer.
That should lead you toward retrieval-based approaches such as RAG, rather than assuming the foundation model needs to be retrained.
Foundation Models Are Not All the Same
"Foundation model" describes a category, not a single type of model.
Different foundation models can have very different capabilities.
A model may be optimized for:
-
Text generation
-
Image generation
-
Speech
-
Code
-
Embeddings
-
Multimodal tasks
-
Other specialized applications
A model designed primarily for generating images isn't automatically the right choice for a customer-service chatbot. Likewise, a text-generation model isn't necessarily the best tool for every AI workload.
This leads to an important practical skill:
Model selection
Choosing a foundation model should be based on the requirements of the application.
Consider factors such as:
Task
What does the application actually need to do?
Modality
Does it work with text, images, audio, video, or multiple types of input?
Quality
How accurate or capable does the model need to be?
Latency
How quickly does the application need a response?
Cost
How much can the organization spend on inference?
Context requirements
How much information must the model handle in a single request?
Safety and governance
What controls are required for the application's use case?
There is rarely a single "best" foundation model.
The right model is the one that best fits the application's requirements.
This is where model selection becomes more than a vocabulary question. AIF-C01 can give you a business requirement and several plausible AI options. The correct answer isn't necessarily the newest, largest, or most capable model.
Pay attention to what the organization actually needs: modality, performance, cost, latency, context requirements, and the intended task.
A scenario may deliberately give you a model that is technically more capable but unnecessarily expensive or slow. If the requirement says the application needs low latency and high-volume inference, for example, capability is only one part of the decision.
AIF-C01 expects you to consider the business requirement, not simply select the most powerful option.
Foundation Models and Amazon Bedrock
For the AWS Certified AI Practitioner exam, foundation models are closely associated with Amazon Bedrock.
Amazon Bedrock provides access to foundation models from multiple providers through AWS services and APIs. This allows developers to build generative AI applications without having to manage the underlying infrastructure required to train and operate the foundation models themselves.
This is an important distinction:
Bedrock is a service for building generative AI applications using foundation models. It is not itself a foundation model.
Bedrock can provide access to different models and capabilities, allowing organizations to select models that fit their particular requirements.
That makes model selection an important part of working with Bedrock.
For AIF-C01, it is particularly important to keep Amazon Bedrock and the foundation models available through Bedrock conceptually separate.
Bedrock is the AWS service used to build generative AI applications and access foundation models; it is not itself a single foundation model. When a question asks which AWS service provides access to foundation models for building generative AI applications, that distinction matters.
The Foundation Model Lifecycle
It helps to put all of these concepts together.
A simplified view looks like this:
Training data
↓
Pre-training
↓
Learned parameters
↓
Foundation model
↓
Inference
↓
Application output
But real-world applications add several layers around that foundation.
An application might provide:
Prompt
↓
Retrieved information
↓
Context
↓
Foundation model
↓
Generated output
The model itself is only one part of the overall application.
This distinction is important because many of the techniques covered on the AIF-C01 exam are about how an application uses a model, rather than how the model was originally trained.
When a question contains several unfamiliar AWS or AI terms, don't try to identify the answer from one keyword. First determine what the scenario is trying to accomplish.
Is it choosing a model?
Supplying external information?
Changing model behavior?
Controlling access?
Evaluating performance?
Protecting users?
Once you identify the underlying requirement, the terminology becomes much easier to sort out.
What Foundation Models Are Good At
Foundation models can provide broad capabilities without requiring an organization to build a separate model for every task.
Depending on the model, applications may use them for:
-
Generating text
-
Summarizing documents
-
Answering questions
-
Translating content
-
Generating or analyzing code
-
Extracting information
-
Classifying content
-
Generating images
-
Processing multiple modalities
-
Conversational applications
The flexibility is one of their greatest advantages.
However, flexibility does not mean that a foundation model should be trusted blindly.
One useful way to approach these questions is to start with the job the organization wants the AI to perform, then work backward toward the technology.
If the requirement is generating or transforming content, look toward generative AI capabilities. If the requirement is a narrowly defined prediction or classification task, a traditional machine learning approach may be more appropriate.
The exam often gives you the business problem first and expects you to recognize the technology that fits it.
What Foundation Models Are Not Good At
A foundation model's output is not automatically correct simply because the model is large or sophisticated.
Models can produce:
-
Incorrect information
-
Biased outputs
-
Unsupported claims
-
Inappropriate content
-
Outdated information
-
Hallucinations
The model also has no inherent understanding that a generated answer is "true" simply because it sounds convincing.
This is why model evaluation, responsible AI, grounding, security, and human oversight matter.
A strong generative AI application isn't simply:
Prompt → Model → Answer
It may also require:
Prompt → Retrieval → Context → Model → Evaluation → Guardrails → Answer → Monitoring
The surrounding application architecture matters enormously.
This is also where several AIF-C01 domains begin to overlap. Foundation models, responsible AI, security, evaluation, and governance aren't isolated subjects.
A question may describe a model producing convincing but incorrect information and expect you to recognize the underlying concern rather than simply identify the model being used. Learning these concepts as connected pieces will serve you better than memorizing each definition independently.
The Exam Perspective
When you see foundation model on the AIF-C01 exam, think:
A broadly trained model that can serve as a reusable foundation for many AI applications and tasks.
When you see pre-training, think:
The large-scale training process that establishes the model's general capabilities.
When you see parameters, think:
Values learned and adjusted during training that allow the model to represent learned patterns.
When you see inference, think:
Using the trained model to generate an output.
When you see tokens, think:
The units of text processed by a language model.
When you see context window, think:
The amount of information the model can consider in a particular interaction.
And when you see a scenario asking you to choose between models, don't automatically choose the largest or most powerful model.
Look at the requirements.
Cost, latency, capability, modality, context requirements, and the specific task all matter.
A useful AIF-C01 strategy is to avoid treating unfamiliar terminology as a vocabulary test. When several answers sound plausible, identify the underlying requirement in the scenario first.
If the question is really about providing current company information, think about retrieval.
If it is about changing how a model behaves, think about customization or prompting.
If it is about selecting an appropriate model, look at capability, modality, latency, cost, and context requirements.
If it is about using a foundation model through AWS to build a generative AI application, consider the role of Amazon Bedrock.
The terminology becomes much easier once you understand what problem each piece is solving.
The Mental Model to Keep
If you remember only one picture, remember this:
Foundation model = the reusable base
Pre-training = how the base learns general capabilities
Parameters = what the training process adjusts
Inference = using the trained model
Prompt = instructions and input
Context = information supplied for the current interaction
RAG = supplying relevant external information
Fine-tuning = changing model behavior through additional training
Application = everything built around the model to solve a real problem
The goal isn't to memorize a chain of definitions such as foundation model → parameters → inference → tokens. The goal is to be able to recognize how those concepts work together when you're given a real-world scenario.
That's much closer to how these concepts become useful on the AIF-C01 exam — and much closer to how you'll encounter them outside the exam as well.