Foundation Models Explained: The Adaptable AI Engines That Go Far Beyond Text

A foundation model is an AI model trained on broad, often unlabeled data that can be adapted to a wide range of downstream tasks — and it is a much bigger category than the chatbots it powers. The term was coined in 2021 by researchers at Stanford University's Center for Research on Foundation Models and Institute for Human-Centered Artificial Intelligence, who described these models as a "paradigm shift." The shift is real: instead of building a narrow, task-specific model for every problem, developers now train one large, general-purpose model once and adapt it many times. Large language models like the GPT family are the most famous examples, but the paradigm extends to images, audio, code, and even robot sensor data.

What are Foundation Models?

The Stanford researchers who named the category defined foundation models as "models trained on broad data (generally using self-supervision at scale) that can be adapted to a wide range of downstream tasks," a definition the Ada Lovelace Institute recounts in its explainer on the technology. The name itself carries the key idea. As the Stanford authors put it, "a foundation model is itself incomplete but serves as the common basis from which many task-specific models are built via adaptation."

That incompleteness is the point. A foundation model is not a finished product; it is a substrate. One pre-trained model becomes the common basis for translation, summarization, question answering, or code generation, each achieved through adaptation rather than training from scratch. Google Cloud characterizes this as the move "from specialized tools to adaptable, general-purpose models," calling it "the hallmark of the foundation model paradigm."

The practical consequences are already visible in everyday products. The Ada Lovelace Institute notes that foundation models form the basis of applications including OpenAI's ChatGPT, Microsoft's Bing, and many website chatbots, and they similarly underpin image generation tools such as Midjourney and Adobe Photoshop's generative fill. ChatGPT, for example, is built on the GPT-3.5 and GPT-4 families of foundation models, which also serve as the base for other applications such as Bing Chat and Duolingo Max.

Foundation Models vs. LLMs and Generative AI

The terms "foundation model," "large language model," and "generative AI" are often used interchangeably, but they sit at different levels of a conceptual hierarchy, and confusing them obscures what the paradigm actually covers.

Start with LLMs. Google Cloud frames the relationship as a parent-child one: "all LLMs are foundation models, but not all foundation models are LLMs." The difference lies in the data. LLMs, as the name implies, are trained specifically on vast amounts of text and code, and they specialize in understanding and generating human language — translation, summarization, chatbot interactions. They are a major type of foundation model, but only one type.

Generative AI sits at a different level entirely. Google Cloud suggests thinking of it as "the 'engine' vs. the 'function'": the foundation model is the pre-trained engine built on massive data and designed for adaptation, while generative AI is a primary function that engine can perform — creating new content like text, images, or code. The distinction matters because not every foundation model is used generatively. A foundation model could be adapted for non-generative purposes such as complex classification or analysis. Most popular foundation models do power generative tasks, which is why the two ideas blur together in casual usage, but the categories are not identical.

Key Characteristics: Self-Supervised Learning and Adaptability

Two technical properties make the paradigm possible, and both distinguish foundation models from the machine learning architectures that preceded them.

The first is self-supervised learning. Amazon Web Services explains that foundation models "use self-supervised learning to create labels from input data," meaning no one has instructed or trained the model with labeled training data sets. The model generates its own training signal from the raw data — predicting the next word in a sequence, for instance, which AWS describes as selecting the next word using probability distribution techniques. This feature, AWS notes, separates these models from previous ML architectures, which use supervised or unsupervised learning. The consequence is enormous: labeling data at scale has always been the bottleneck in machine learning, and self-supervision removes it, letting models learn from mountains of raw, unlabeled material.

The second is adaptability. Because the model learns broad patterns from its training data, the same underlying network can be adapted to many downstream tasks — the transfer of learned patterns from one task to another. AWS adds that even after pre-training, foundation models "can continue to learn from data inputs or prompts during inference," meaning developers can shape comprehensive outputs through carefully curated prompts without retraining the model. NVIDIA's explainer captures the umbrella nature of the category, noting that researchers identified transformer models, large language models, vision language models, and other neural networks as part of this new class, defined by an AI neural network "trained on mountains of raw data" that "can be adapted to accomplish a broad range of tasks."

Foundation Models Across Modalities

The clearest way to see the paradigm's breadth is to sort foundation models by the kind of data they process. The table below categorizes the major types described in the evidence, showing how the same principles — self-supervised pre-training on broad data, then adaptation — recur across text, images, audio, and physical sensor data.

Foundation model types compared by primary data modality, core function, and example application
Model Type / Example	Primary Modality	Core Function	Example Application
Large language models (GPT-3.5, GPT-4, Med-PaLM 2)	Text and code	Understand and generate human language; excel at translation, summarization, and chatbot interaction	ChatGPT, Bing Chat, Duolingo Max; Med-PaLM 2 answers medical questions
Multimodal models	Text, images, and audio together	Analyze and generate content across multiple modalities, integrating understanding across data forms	Google is designing a multimodal version of Med-PaLM 2 that synthesizes information from medical images
Embodied / robotics models (PaLM-E)	Robot sensor data, plus language and visual inputs	Transfer knowledge from language and visual domains to robotics, helping robots adapt to new environments and generalize across tasks and machine embodiments	PaLM-E, an embodied multimodal language model trained on robot sensor data
Code generation models	Text (programming languages)	Complete, debug, explain, and generate code in different programming languages	Developer coding assistance built on foundation models
Generative adversarial networks (GANs)	Varies by training data	Two neural networks contest with each other in a zero-sum game to produce outputs	Image generation pipelines

Foundation model types compared by primary data modality, core function, and example application

Model Type / Example

Primary Modality

Core Function

Example Application

Large language models (GPT-3.5, GPT-4, Med-PaLM 2)

Text and code

Understand and generate human language; excel at translation, summarization, and chatbot interaction

ChatGPT, Bing Chat, Duolingo Max; Med-PaLM 2 answers medical questions

Multimodal models

Text, images, and audio together

Analyze and generate content across multiple modalities, integrating understanding across data forms

Google is designing a multimodal version of Med-PaLM 2 that synthesizes information from medical images

Embodied / robotics models (PaLM-E)

Robot sensor data, plus language and visual inputs

Transfer knowledge from language and visual domains to robotics, helping robots adapt to new environments and generalize across tasks and machine embodiments

PaLM-E, an embodied multimodal language model trained on robot sensor data

Code generation models

Text (programming languages)

Complete, debug, explain, and generate code in different programming languages

Developer coding assistance built on foundation models

Generative adversarial networks (GANs)

Varies by training data

Two neural networks contest with each other in a zero-sum game to produce outputs

Image generation pipelines

Reading the table this way makes the argument of the paradigm concrete. Each row is a different data modality, yet each relies on the same underlying recipe: pre-train broadly, then adapt. IBM's overview places these models in expansive domains including computer vision, natural language processing, and speech recognition. The robotics row is perhaps the most striking illustration that this is not merely a language technology — PaLM-E extends the foundation model approach to physical-world interaction and control, trained on sensor data rather than prose.

One caveat on scope: this table uses representative examples to illustrate the concept, not a comprehensive catalog of every foundation model in existence. The category is deliberately broad, and new variants appear continuously.