transformer architecture
3 episodes mention this concept
lexfridmanFeb 3, 2025DeepSeek, OpenAI, and the Geopolitics of AI: A Technical Dive into Open Weights, Reasoning Models, and Efficiency
TechnologyMixture of Experts (MoE)Open WeightsChain of Thought ReasoningPre-training- lexfridmanJan 17, 2019
Deep Learning State of the Art (2019): Breakthroughs in NLP, Applied AI, and Reinforcement Learning
TechnologyDeep Learning State of the ArtNatural Language Processing (NLP)Encoder-Decoder ArchitectureRecurrent Neural Networks (RNNs) - lexfridman
Oriol Vinyals on AlphaStar, Deep Reinforcement Learning, and the Evolution of AI in Real-Time Strategy Games
TechnologyDeep LearningReinforcement Learning (RL)AlphaStarStarCraft (Real-Time Strategy game)
Knowledge Graph
Related concepts — line thickness indicates connection strength. Click any node to explore.
Related concepts
Concepts that appear alongside transformer architecture across episodes.
- synthetic data
- dense model
- open source ai
- autor-regressive prediction
- natural language processing (nlp)
- splash damage
- recurrent neural networks (rnns)
- mixture of experts (moe)
- action space
- dilated/atrous convolutions
- self-attention
- neural machine translation
- reinforcement learning (rl)
- scaling laws
- ai megaclusters
- video-to-video synthesis
- battle.net
- meta emotion
- world of warcraft
- cuda layer