Foundation Models for Recommendations
How Meta's GEM learns from billions of interactions to power the next generation of recommendation systems.
Introduction
Recommendation systems sit at the heart of modern advertising. Their job is to match the right ad to the right user at the right time. Traditionally, this has been achieved through a collection of specialized models that learn from user demographics, interests, engagement history, and advertiser signals to predict which ads are most relevant.
Over the past decade, recommendation systems have evolved from feature-engineered ranking models to deep learning systems capable of capturing increasingly complex patterns in user behavior.
Inspired by the success of large language models (LLMs), Meta is rethinking ad recommendation as a foundation model problem. Its Generative Ads Recommendation Model (GEM) represents a move toward a single large-scale model that learns broadly from user interactions and transfers that knowledge across the advertising ecosystem.
How GEM Works
At its core, GEM learns from sequences of user behavior rather than relying primarily on static user profiles or handcrafted features. Instead of asking, “Who is this user?”, GEM focuses on understanding, “What has this user been doing, and what are they likely to do next?”
This sequence-based approach allows the model to capture evolving interests and contextual signals across billions of interactions on Facebook and Instagram. By learning rich representations of users, ads, and engagement patterns, GEM acts as a central intelligence layer that can support multiple downstream ranking and recommendation tasks.
Much like an LLM predicts the next word in a sentence, GEM learns to predict future user actions from sequences of behavioral events, enabling more accurate and adaptive recommendations.
Innovations in Training GEM
What makes GEM particularly interesting is not only its scale but also how it was trained. Meta applied several principles that have become common in frontier AI systems:
Large-scale pretraining across massive user interaction datasets.
Distributed training across thousands of GPUs.
Advanced sequence modeling architectures optimized for recommendation workloads.
Knowledge distillation techniques that transfer what GEM learns into smaller production models.
Meta also introduced infrastructure innovations, including optimized GPU kernels, memory-efficient training strategies, and improved parallelization methods that significantly increased training efficiency.
Perhaps the most important innovation is the shift from training many independent recommendation models toward training a single foundational model that can teach and improve an entire ecosystem of downstream systems.
Meta reports that GEM improved ad conversions across Facebook and Instagram, validating the use of foundation-model techniques in recommendation systems.
More broadly, GEM signals a shift toward training recommendation systems like LLMs, where a single foundation model learns from massive interaction data and powers personalization across multiple downstream tasks.