Mixture-of-experts models

Trend over time

Evidence papers — 50 most recent of 732

• IntBMoE: Integrating Block-Level Conditioning into Expert Composition for Full-Participation Mixture-of-Experts
  Ran Cheng, Longfei Xu, Zheng Liu, Kaikui Liu, Xiangxiang Chu · Sep 2026 · arXiv
• Trading Depth for Time in Recurrent Transformers
  Zeyi Huang, Xuehai He, Yong Jae Lee, Yelong Shen · Sep 2026 · arXiv
• Accelerating Dense LLMs via L0-regularized Mixture-of-Experts
  Zhenyu Zhang, Jiudong Yang, Zhaowen Tao, Meng Chen · Sep 2026 · arXiv
• WS-NeRF: A Mamba-Driven World-State-Aware Adaptive Deblurring Neural Radiance Field
  Hang Jiang, Jinghao Wang, Yiming Zhang, Xinhong Wang, Luwei Ran, Yinfeng Yu · Sep 2026 · arXiv
• OneBid: A Unified Auto-Bidding Foundation Model for Diverse oCPX Advertising Scenarios
  Yewen Li, Peng Jiang, Yitian Li, Pengfei Lv, Xialong Liu, Peng Jiang, Qingpeng Cai · Sep 2026 · arXiv
• Attention-Aware Routing: Coupling Routing and Attention in MoEs
  Despoina Kosmopoulou, Anastasios Tsetsilas, Efthymios Georgiou, Giannis Karamanolakis, Swastik Roy, Alexandros Potamianos · Sep 2026 · arXiv
• OceanMoE: Structured Conditional Sparse Computation for Long-Horizon Multivariate Ocean Forecasting
  Yishun Zhu, Jian Wang · Sep 2026 · arXiv
• DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
  DeepSeek-AI, :, Anyi Xu, B. Li, Bangcai Lin, Bing Xue, BingCheng Xian, Bingzheng Xu, Bochao Wu, Bowei Zhang, Boyi Deng, C. C. Yu, Chao Jin, Chaofan Lin, Chen Dong, Chenbing Wang, Chenfan Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chengyuan Zhang, Chenhao Xu, Chenqi Zhao, Chenze Shao, Chuhao Wang, Chuqi Zhang, Damai Dai, Dejian Yang, Deli Chen, Di Huang, Di Wu, Donghao Li, Erhang Li, Eric Fu, F. Zhou, Fangwei Zhou, Fangyun Lin, Fangzhou Yuan, Feiyu Xia, Fucong Dai, Guangbo Hao, Guanglin Li, · Sep 2026 · arXiv
• Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data
  Jinli Hu, Ross M. Clarke, Yichuan Zhang, José Miguel Hernández-Lobato · Sep 2026 · arXiv
• CoAtNet-DeepMoE: A Convolution-Attention Hybrid with DeepSeek Mixture-of-Experts for Parameter-Efficient Tomato Disease Classification
  Md Nadim Mahamood, Md Arif Shahriar, Md Shafi Ud Doula, Kamrul Hasan · Sep 2026 · arXiv
• StrucPhysVideo: Learning Physical Dynamics from Structured Captions and Robot Actions
  Awomo-WM Team, :, Enhui Ma, Kaiwen Guo, Tingrui Zhang, Wei Song, Yingshui Tan, Jianhua Xu, Tong Zhang, Kaicheng Yu · Sep 2026 · arXiv
• Occluded Gait Recognition with Mixture of Experts: An Action Detection Perspective
  Panjian Huang, Yunjie Peng, Saihui Hou, Chunshui Cao, Xu Liu, Zhiqiang He, Yongzhen Huang · Sep 2026 · arXiv
• Generalist-Specialist Mixture-of-Experts for Rare Pathology Detection in Multimodal Imaging
  Johannes Kaiser, Florian Braunmiller, Daniel Rückert, Georgios Kaissis · Sep 2026 · arXiv
• Higher-order pruning of experts in mixture-of-experts language models
  Alex M. Tseng, Prannay Kaul, Luca Zancato, Wei Xia, Stefano Soatto · Sep 2026 · arXiv
• Colla-Q: Toward Collaborative Experts in MoE Quantization via Minimax Precision Balancing
  Eunju Shin, Jongbin Ryu · Sep 2026 · arXiv
• The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction
  Yu Lin, Yiming Wang, Runyuan Cai, Hanze Liu, Xiaodong Zeng · Sep 2026 · arXiv
• MoRE: Mixture of Reused Experts
  Eric S. Qiu, Utku Umur Acikalin, Justin Lovelace, Christian Belardi, Arjun B. Mulchandani, Carla P. Gomes, Kilian Q. Weinberger · Sep 2026 · arXiv
• Multi-View Mixture-of-Experts with Vision-Language Reranking for Cross-View Object Geo-Localization
  Xuyu Fan, Qi Ming, Zhu Han, Liuqian Wang, Siyuan Cao, Xiaohan Zhang, Xudong Zhao, Mingjing Zhao, Yuhan Zhang · Sep 2026 · arXiv
• Multi-Appliance Non-Intrusive Load Monitoring via Label-Preserving Aggregate Recomposition and Prediction Consistency
  Jiangfeng Liu, Yanfang Fan · Sep 2026 · arXiv
• SOTER: A Generative Time-Series Foundation Model for Wearable Human Physiological Signals
  Fangke Chen, Sirry Chen, Wei Chen, Zhongyu Wei · Sep 2026 · arXiv
• Beyond the Previous Layer: Residual Predictive Structure in Sparse MoE Routing
  Hao Li, Yasuyuki Tahara, Yuichi Sei · Sep 2026 · arXiv
• Unifying Semantic Priors and High-Frequency Traces: Enhancing V-JEPA with Mixture-of-Experts for Robust Synthetic Image Forensics
  Simone Teglia, Irene Amerini · Sep 2026 · arXiv
• What Breaks Under Pruning in Smart Homes, and When? Evaluating LLM Degradation Across Architectures and Task Complexity
  Congjing Zhang, Vashishtha Patil, Henning Lange, Usman Aleem · Sep 2026 · arXiv
• The Misery of Mechanistic Interpretability: A Formal Perspective
  Tobias Ladner, Matthias Althoff · Sep 2026 · arXiv
• MoME: Mixture-of-Memory Embeddings for Context-Aware Sparse Lookup
  Muchen Li, Leonid Sigal, Renjie Liao · Sep 2026 · arXiv
• STHMoE: Hypergraph-Enhanced Heterogeneous Dependency Coordination for LLM-Based Urban Traffic Data Forecasting
  Jiawen Chen, Qi Shao, Yongjian Chang, Mingtong Zhou, Duxin Chen, Wenwu Yu · Sep 2026 · arXiv
• Metacognitive Steering: Learning the Structure of Scientific Judgment
  Vincent Karpf, Joseph Reth, Eike Gerhardt, Audrey Wang, Anna Butz, Jiehao Xing, Jialing Song, Larry Callahan · Sep 2026 · arXiv
• Multimodal deep learning from spectra for small-molecule structure identification: enhancing robustness with mixed-condition training
  Bowen Gao, Lei Zhu, Yiying Wang, Wenjie Yu · Sep 2026 · arXiv
• Physically Partitioned KVCache Format for CPU--GPU Load Balancing in MoE Inference
  Enda Yu, Dezun Dong, Xiangke Liao · Sep 2026 · arXiv
• LLaTSA: Large Language Model-Aligned General-Purpose Transient Stability Analysis
  Chao Shen, Hongwei Zhen, Junyan Shao, Zhenghao Yang, Yifan Zhang, Mingyang Sun · Sep 2026 · arXiv
• North Small Translate: Advanced Cost-Effective Translation (Cohere CAT+)
  Tom Kocmi, Alexandre Bérard, Phil Blunsom, Samuel Cahyawijaya, Shaun Cassini, Nicholas Frosst, Ona de Gibert, Aidan Gomez, Nithya Govindarajan, Shun Kiyono, Olivia Lasche, Lawrence Rogers, Kelly Marchisio, Nikita Moghe, Yash More, Camila Moran-Hidalgo, Yiyang Nan, Michael Sachs, Trisha Starostina, Daan van Stigt, Spencer Rarrick, Sebastian Vincent, Ivan Zhang · Sep 2026 · arXiv
• A Multi-Resolution Multi-Domain Pre-Training Framework for Universal Traffic Forecasting
  Zhouyang Liu, Jindong Han, Hao Wang, Xinyue Liu, Hui Gao, Dongsheng Li, Hao Liu · Sep 2026 · arXiv
• Multi-Modal Tumor Survival Prediction via Graph-Guided Mixture of Experts
  H Mathavan, H Liu · Sep 2026 · arXiv
• Quality-Constrained Routing over a Fixed Pool of Quantized Mixture-of-Experts Instances
  Zhenghong Huang, Hongfan Wu, Jiheng Zhang · Sep 2026 · arXiv
• ExpertHTR: Unified Handwritten Text Recognition with Multi-Task Learning and Sparse Mixture-of-Experts
  Dang Hoai Nam, Nguyen Duy Hieu, Quang Huu Hieu, Vo Nguyen Le Duy · Sep 2026 · arXiv
• Task-Aware Federated Fine-Tuning for MoE-based Large Language Models
  Tingqi Wang, Hongyu Ke, Haoxin Wang, Rafal Angryk, Zhipeng Cai · Sep 2026 · arXiv
• SeqMoE: Toward Full-Load Performance via Predictive and Graph-Compatible MoE Offloading
  Zihan Wang, Yuqi Wang, Lei Gong, Cheng Tang, Wenqi Lou, Teng Wang, Chao Wang, Xuehai Zhou · Sep 2026 · arXiv
• Mixture-of-Experts Language Models Can Be Strong and Efficient Retrievers
  Anubhav Shrestha, Safal Shrestha, Minwu Kim, Torsten Suel, Keith Ross · Sep 2026 · arXiv
• Expert-Space Exploration in MoE Reinforcement Learning
  Hongyi He, Zhenghao Lin, Xiao Liu, Peng Cheng, Yan Lu, Yeyun Gong · Sep 2026 · arXiv
• T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks
  Junyao Yang, Yucheng Shi, Zhongzhi Li, Ruhan Wang, Zongxia Li, Haitao Mi, Leowei Liang · Sep 2026 · arXiv
• Pelican-Sim 1.0: A General World Model Simulator for Embodied Intelligence
  Shilong Zou, Shilin Zhang, Yingji Zhang, Yuhang Huang, Yi Zhang, Zeyuan Ding, Han Dong, Junwei Liao, Yong Dai, Jian Tang, Xiaozhu Ju · Sep 2026 · arXiv
• Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
  Atindra Jha, Margaret Li, Jure Leskovec, Percy Liang, Luke Zettlemoyer · Sep 2026 · arXiv
• Characterizing Job Power Elasticity for Power-Flexible AI Training
  Philip Colangelo, Charles Dawson, Shayan Sengupta, Ayse Coskun, Varun Sivaram · Sep 2026 · arXiv
• Breaking the Central Bias: Spatially Partitioned Experts for Coordinate-Based Neuroevolution
  Romain Claret, Arthur Gygax, Michael O'Neill, Paul Cotofrei, Michael Palma Mendes, Pascal Felber · Sep 2026 · arXiv
• A Trust-Network-Based Federated Learning Framework for Multi-Center Aging Clock Prediction
  Chunxu Zhang, Bo Li, Wenliang Wang, Yang Liu, Di Jiang, Yuan Huang, Yo-ichi Nabeshima, Akinori Yamamura, Bo Yang, Qiang Yang · Sep 2026 · arXiv
• How Fragile Is Safety Alignment at Frontier Scale? A Single-Direction Attack on a 320B MoE
  Yi Shi, Tanyu Chen, Kai Shen · Sep 2026 · arXiv
• FreqFLD: Towards All-in-One Facial Landmark Detection via Frequency Modulation
  Shun Ren, Kaijie Jin, Shengkai Hu, Beihang Song, Hang Sun, Wenwen Min, Youfa Liu, Jun Wan · Sep 2026 · arXiv
• From Semantic to Token Communication: The Next Paradigm for Large-Model-Driven 6G Intelligent Connectivity
  Yu Ma, Zhen Gao, Li Qiao, Xiaoyuan Zhang, Mahdi Boloursaz Mashhadi, Yin Xu, Wenjun Xu, Xiaodong Xu, Kaibin Huang, Jiangzhou Wang, Rahim Tafazolli, Sheng Chen, Tony Q. S. Quek, Ping Zhang · Sep 2026 · arXiv
• Chimaera: A Mixture-of-Graph-Experts Architecture for Cross-Task and Cross-Dataset Graph Learning
  Jonathan Frank, David Richerby, Ansgar Scherp · Sep 2026 · arXiv
• Nearly Tight Rademacher Bounds for Sparsely Activated Neural Networks
  Xiaoyu Li, Zhizhou Sha, Jiaojiao Jiang, Junbin Gao, Andi Han · Sep 2026 · arXiv