InfiniPipe: Elastic Pipeline Parallelism for Efficient Variable-Length Long-Context LLM Training

Abstract: Long context training is crucial for LLM's context extension. Existing schemes, such as sequence parallelism, incur substantial communication overhead. Pipeline parallelism (PP) reduces this cost, but its effectiveness hinges on partitioning granularity. Batch-level PP employing sequence packing exhibits high memory consumption in long-context scenarios, whereas token-level PP splitting sequences into slices alleviates memory overhead but may incur hardware under-utilization. Moreover, the skewed distribution of sequence length in real-world datasets renders monolithic and static granularity PP's sub-optimal performance. In this paper, we propose 1) \textit{Elastic Pipeline Parallelism} (EPP) that orchestrates token-level PP and batch-level PP to adapt to resource and workload heterogeneity, and 2) \textit{Stage-Aware Chunk-Level Adaptive Checkpointing} that efficiently integrates gradient checkpointing with EPP. Comprehensive experiments demonstrate that InfiniPipe achieves a 1.69x speedup over state-of-the-art systems. Our code is open-sourced at https://github.com/wsjdsg/InfiniPipe-code.git.

Shiju Wang
Yujie Wang
Ao Sun
Fangcheng Fu
Zijian Zhu
Bin Cui
Xu Han
Kaisheng Ma


Steering Multimodal Large Language Models Decoding for Context-Aware Safety

Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed in real-world applications, yet their ability to make context-aware safety decisions remains limited. Existing methods often fail to balance oversensitivity (unjustified refusals of benign queries) and undersensitivity (missed detection of visually grounded risks), leaving a persistent gap in safety alignment. To address this issue, we introduce Safety-aware Contrastive Decoding (SafeCoDe), a lightweight and model-agnostic decoding framework that dynamically adjusts token generation based on multimodal context. SafeCoDe operates in two stages: (1) a contrastive decoding mechanism that highlights tokens sensitive to visual context by contrasting real and Gaussian-noised images, and (2) a global-aware token modulation strategy that integrates scene-level reasoning with token-level adjustment to adapt refusals according to the predicted safety verdict. Extensive experiments across diverse MLLM architectures and safety benchmarks, covering undersensitivity, oversensitivity, and general safety evaluations, show that SafeCoDe consistently improves context-sensitive refusal behaviors while preserving model helpfulness.

Zheyuan Liu
Zhangchen Xu
Guangyao Dou
Xiangchi Yuan
Zhaoxuan Tan
Radha Poovendran
Meng Jiang


CompLLM: Compression for Long Context Q&A

Abstract: Large Language Models (LLMs) face significant computational challenges when processing long contexts due to the quadratic complexity of self-attention. While soft context compression methods, which map input text to smaller latent representations, have shown promise, their real-world adoption is limited. Existing techniques typically compress the context as a single unit, which leads to quadratic compression complexity and an inability to reuse computations across queries with overlapping contexts. In this work, we introduce CompLLM, a soft compression technique designed for practical deployment. Instead of processing the context holistically, CompLLM divides it into segments and compresses each one independently. This simple design choice yields three critical properties: efficiency, as the compression step scales linearly with the context length; scalability, enabling models trained on short sequences (e.g., 1k tokens) to generalize to contexts of 100k tokens; and reusability, allowing compressed segments to be cached and reused across different queries. Our experiments show that with a 2x compression rate, at high context lengths CompLLM speeds up Time To First Token (TTFT) by up to 4x and reduces the KV cache size by 50%. Furthermore, CompLLM achieves performance comparable to that obtained with the uncompressed context, and even surpasses it on very long sequences, demonstrating its effectiveness and practical utility.

Gabriele Berton
Jayakrishnan Unnikrishnan
Son Tran
Mubarak Shah


Towards Adaptive Context Management for Intelligent Conversational Question Answering

Abstract: This particular paper introduces an Adaptive Context Management (ACM) framework for the Conversational Question Answering (ConvQA) systems. The key objective of the ACM framework is to optimize the use of the conversation history by dynamically managing context for maximizing the relevant information provided to a ConvQA model within its token limit. Our approach incorporates a Context Manager (CM) Module, a Summarization (SM) Module, and an Entity Extraction (EE) Module in a bid to handle the conversation history efficaciously. The CM Module dynamically adjusts the context size, thereby preserving the most relevant and recent information within a model's token limit. The SM Module summarizes the older parts of the conversation history via a sliding window. When the summarization window exceeds its limit, the EE Module identifies and retains key entities from the oldest conversation turns. Experimental results demonstrate the effectiveness of our envisaged framework in generating accurate and contextually appropriate responses, thereby highlighting the potential of the ACM framework to enhance the robustness and scalability of the ConvQA systems.

Manoj Madushanka Perera
Adnan Mahmood
Kasun Eranda Wijethilake
Quan Z. Sheng

International Conference on Advanced Data Mining and Applications


Context Is What You Need: The Maximum Effective Context Window for Real World Limits of LLMs

Abstract: Large language model (LLM) providers boast big numbers for maximum context window sizes. To test the real world use of context windows, we 1) define a concept of maximum effective context window, 2) formulate a testing method of a context window's effectiveness over various sizes and problem types, and 3) create a standardized way to compare model efficacy for increasingly larger context window sizes to find the point of failure. We collected hundreds of thousands of data points across several models and found significant differences between reported Maximum Context Window (MCW) size and Maximum Effective Context Window (MECW) size. Our findings show that the MECW is, not only, drastically different from the MCW but also shifts based on the problem type. A few top of the line models in our test group failed with as little as 100 tokens in context; most had severe degradation in accuracy by 1000 tokens in context. All models fell far short of their Maximum Context Window by as much as 99 percent. Our data reveals the Maximum Effective Context Window shifts based on the type of problem provided, offering clear and actionable insights into how to improve model accuracy and decrease model hallucination rates.

Advances in Artificial Intelligence and Machine Learning


LiteLong: Resource-Efficient Long-Context Data Synthesis for LLMs

Abstract: High-quality long-context data is essential for training large language models (LLMs) capable of processing extensive documents, yet existing synthesis approaches using relevance-based aggregation face challenges of computational efficiency. We present LiteLong, a resource-efficient method for synthesizing long-context data through structured topic organization and multi-agent debate. Our approach leverages the BISAC book classification system to provide a comprehensive hierarchical topic organization, and then employs a debate mechanism with multiple LLMs to generate diverse, high-quality topics within this structure. For each topic, we use lightweight BM25 retrieval to obtain relevant documents and concatenate them into 128K-token training samples. Experiments on HELMET and Ruler benchmarks demonstrate that LiteLong achieves competitive long-context performance and can seamlessly integrate with other long-dependency enhancement methods. LiteLong makes high-quality long-context data synthesis more accessible by reducing both computational and data engineering costs, facilitating further research in long-context language training.

Junlong Jia
Xing Wu
Chaochen Gao
Ziyang Chen
Zijia Lin
Zhongzhi Li
Weinong Wang
Haotian Xu
Donghui Jin
Debing Zhang
Binghui Guo

AAAI Conference on Artificial Intelligence


Q-ROAR: Outlier-Aware Rescaling for RoPE Position Interpolation in Quantized Long-Context LLMs

Abstract: Extending LLM context windows is crucial for long range tasks. RoPE-based position interpolation (PI) methods like linear and frequency-aware scaling extend input lengths without retraining, while post-training quantization (PTQ) enables practical deployment. We show that combining PI with PTQ degrades accuracy due to coupled effects long context aliasing, dynamic range dilation, axis grid anisotropy, and outlier shifting that induce position-dependent logit noise. We provide the first systematic analysis of PI plus PTQ and introduce two diagnostics: Interpolation Pressure (per-band phase scaling sensitivity) and Tail Inflation Ratios (outlier shift from short to long contexts). To address this, we propose Q-ROAR, a RoPE-aware, weight-only stabilization that groups RoPE dimensions into a few frequency bands and performs a small search over per-band scales for W_Q,W_K, with an optional symmetric variant to preserve logit scale. The diagnostics guided search uses a tiny long-context dev set and requires no fine-tuning, kernel, or architecture changes. Empirically, Q-ROAR recovers up to 0.7% accuracy on standard tasks and reduces GovReport perplexity by more than 10%, while preserving short-context performance and compatibility with existing inference stacks.


HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models

Abstract: Positional encoding mechanisms enable Transformers to model sequential structure and long-range dependencies in text. While absolute positional encodings struggle with extrapolation to longer sequences due to fixed positional representations, and relative approaches like Alibi exhibit performance degradation on extremely long contexts, the widely-used Rotary Positional Encoding (RoPE) introduces oscillatory attention patterns that hinder stable long-distance dependency modelling. We address these limitations through a geometric reformulation of positional encoding. Drawing inspiration from Lorentz transformations in hyperbolic geometry, we propose Hyperbolic Rotary Positional Encoding (HoPE), which leverages hyperbolic functions to implement Lorentz rotations on token representations. Theoretical analysis demonstrates that RoPE is a special case of our generalized formulation. HoPE fundamentally resolves RoPE's slation issues by enforcing monotonic decay of attention weights with increasing token distances. Extensive experimental results, including perplexity evaluations under several extended sequence benchmarks, show that HoPE consistently exceeds existing positional encoding methods. These findings underscore HoPE's enhanced capacity for representing and generalizing long-range dependencies. Data and code will be available.

Chang Dai
Hongyu Shan
Mingyang Song
Di Liang


The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management

Abstract: Large Language Model (LLM)-based agents solve complex tasks through iterative reasoning, exploration, and tool-use, a process that can result in long, expensive context histories. While state-of-the-art Software Engineering (SE) agents like OpenHands or Cursor use LLM-based summarization to tackle this issue, it is unclear whether the increased complexity offers tangible performance benefits compared to simply omitting older observations. We present a systematic comparison of these approaches within SWE-agent on SWE-bench Verified across five diverse model configurations. Moreover, we show initial evidence of our findings generalizing to the OpenHands agent scaffold. We find that a simple environment observation masking strategy halves cost relative to the raw agent while matching, and sometimes slightly exceeding, the solve rate of LLM summarization. Additionally, we introduce a novel hybrid approach that further reduces costs by 7% and 11% compared to just observation masking or LLM summarization, respectively. Our findings raise concerns regarding the trend towards pure LLM summarization and provide initial evidence of untapped cost reductions by pushing the efficiency-effectiveness frontier. We release code and data for reproducibility.

Tobias Lindenbauer
Igor Slinko
Ludwig Felder
Egor Bogomolov
Yaroslav Zharov


Joint Enhancement of Relational Reasoning for Long-Context LLMs

Abstract: Despite significant progress, large language models (LLMs) still struggle with long contexts due to memory limitations and their inability to tackle complex and long-context tasks. Additionally, LLMs often suffer from a lack of transparency and are prone to producing hallucinations. To address these challenges, we propose \textbf{JERR}, a novel framework designed to enhance long-context comprehension via graph-based reasoning in LLMs. JERR integrates three key components: synopsis extraction, graph construction, and relational reasoning. First, synopsis is extracted by chunking text strategically, allowing the model to summarize and understand information more efficiently. Second, we build a directed acyclic graph (DAG) to resolve redundancy, ensuring logical consistency and clarity. Finally, we incorporate Monte Carlo Tree Search (MCTS) to help the model navigate complex reasoning paths, ensuring more accurate and interpretable outputs. This framework provides a novel solution that enables LLMs to handle extended contexts and complex reasoning tasks with improved reliability and transparency. Experimental results show that JERR consistently outperforms all baselines on the ROUGE and F1 metrics, achieving the highest scores on the LLM-Rater evaluation.

Zhirui Chen
Wei Shen
Jiashui Huang
Ling Shao

Conference on Empirical Methods in Natural Language Processing


Strata: Hierarchical Context Caching for Long Context Language Model Serving

Abstract: Large Language Models (LLMs) with expanding context windows face significant performance hurdles. While caching key-value (KV) states is critical for avoiding redundant computation, the storage footprint of long-context caches quickly exceeds GPU memory capacity, forcing production systems to adopt hierarchical caching across memory hierarchies. However, transferring large cached contexts back to the GPU introduces severe performance bottlenecks: fragmented I/O from paged layouts prevents full bandwidth utilization, and existing schedulers fail to account for cache-loading delays, leaving systems loading-bound rather than compute-bound. We present Strata, a hierarchical context caching framework designed for efficient long context LLM serving. Strata introduces GPU-assisted I/O to combat KV cache fragmentation, decoupling GPU and CPU memory layouts and employs cache-aware request scheduling to balance compute with I/O latency and overlapping unavoidable stalls with complementary tasks. Built on SGLang and deployed in production, Strata achieves up to 5x lower Time-To-First-Token (TTFT) compared to vLLM + LMCache and 3.75x speedup over NVIDIA TensorRT-LLM on long-context benchmarks, without degrading short-context performance.

Zhiqiang Xie
Ziyi Xu
Mark Zhao
Yuwei An
Vikram Sharma Mailthody
Scott Mahlke
Michael Garland
Christos Kozyrakis


Agri-Query: A Case Study on RAG vs. Long-Context LLMs for Cross-Lingual Technical Question Answering

Abstract: We present a case study evaluating large language models (LLMs) with 128K-token context windows on a technical question answering (QA) task. Our benchmark is built on a user manual for an agricultural machine, available in English, French, and German. It simulates a cross-lingual information retrieval scenario where questions are posed in English against all three language versions of the manual. The evaluation focuses on realistic "needle-in-a-haystack" challenges and includes unanswerable questions to test for hallucinations. We compare nine long-context LLMs using direct prompting against three Retrieval-Augmented Generation (RAG) strategies (keyword, semantic, hybrid), with an LLM-as-a-judge for evaluation. Our findings for this specific manual show that Hybrid RAG consistently outperforms direct long-context prompting. Models like Gemini 2.5 Flash and the smaller Qwen 2.5 7B achieve high accuracy (over 85%) across all languages with RAG. This paper contributes a detailed analysis of LLM performance in a specialized industrial domain and an open framework for similar evaluations, highlighting practical trade-offs and challenges.


Long-Context Speech Synthesis with Context-Aware Memory

Abstract: In long-text speech synthesis, current approaches typically convert text to speech at the sentence-level and concatenate the results to form pseudo-paragraph-level speech. These methods overlook the contextual coherence of paragraphs, leading to reduced naturalness and inconsistencies in style and timbre across the long-form speech. To address these issues, we propose a Context-Aware Memory (CAM)-based long-context Text-to-Speech (TTS) model. The CAM block integrates and retrieves both long-term memory and local context details, enabling dynamic memory updates and transfers within long paragraphs to guide sentence-level speech synthesis. Furthermore, the prefix mask enhances the in-context learning ability by enabling bidirectional attention on prefix tokens while maintaining unidirectional generation. Experimental results demonstrate that the proposed method outperforms baseline and state-of-the-art long-context methods in terms of prosody expressiveness, coherence and context inference cost across paragraph-level speech.

Zhipeng Li
Xiaofen Xing
Jingyuan Xing
Hangrui Hu
Heng Lu
Xiangmin Xu


Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization

Abstract: Long-context modeling is critical for a wide range of real-world tasks, including long-context question answering, summarization, and complex reasoning tasks. Recent studies have explored fine-tuning Large Language Models (LLMs) with synthetic data to enhance their long-context capabilities. However, the effectiveness of such approaches is often limited by the low diversity and factual inconsistencies in the generated data. To address these challenges, we propose LongMab, a novel framework that leverages a Multi-Armed Bandit (MAB) rollout strategy to identify the most informative chunks from the given long context for sampling high-quality and diverse responses and constructing preference data pairs for Direct Preference Optimization (DPO) training. Specifically, we treat context chunks as arms of MAB, select chunks based on their expected reward scores to input into LLMs to generate responses, and iteratively update these scores based on reward feedback. Both exploration and exploitation during the rollout process enable the LLM to focus on the most relevant context segments, thereby generating and collecting high-quality and diverse responses. Experimental results on both Llama and Qwen show the effectiveness of LongMab by achieving more than a 4% improvement on long-context reasoning benchmarks. All data and code will be released on https://github.com/NEUIR/LongMab-PO.

Shaohua Duan
Pengcheng Huang
Xinze Li
Zhenghao Liu
Xiaoyuan Yi
Yukun Yan
Shuo Wang
Yu Gu
Ge Yu
Maosong Sun


ENCODE: Breaking the Trade-Off Between Performance and Efficiency in Long-Term User Behavior Modeling

Abstract: Long-term user behavior sequences are a goldmine for businesses to explore users' interests to improve Click-Through Rate. However, it is very challenging to accurately capture users' long-term interests from their long-term behavior sequences and give quick responses from the online serving systems. To meet such requirements, existing methods "inadvertently" destroy two basic requirements in long-term sequence modeling: R1) make full use of the entire sequence to keep the information as much as possible; R2) extract information from the most relevant behaviors to keep high relevance between learned interests and current target items. The performance of online serving systems is significantly affected by incomplete and inaccurate user interest information obtained by existing methods. To this end, we propose an efficient two-stage long-term sequence modeling approach, named as EfficieNt Clustering based twO-stage interest moDEling (ENCODE), consisting of offline extraction stage and online inference stage. It not only meets the aforementioned two basic requirements but also achieves a desirable balance between online service efficiency and precision. Specifically, in the offline extraction stage, ENCODE clusters the entire behavior sequence and extracts accurate interests. To reduce the overhead of the clustering process, we design a metric learning-based dimension reduction algorithm that preserves the relative pairwise distances of behaviors in the new feature space. While in the online inference stage, ENCODE takes the off-the-shelf user interests to predict the associations with target items. Besides, to further ensure the relevance between user interests and target items, we adopt the same relevance metric throughout the whole pipeline of ENCODE. The extensive experiment and comparison with SOTA have demonstrated the effectiveness and efficiency of our proposed ENCODE.

Wenji Zhou
Yuhang Zheng
Yinfu Feng
Yunan Ye
Rong Xiao
Long Chen
Xiaosong Yang
Jun Xiao

IEEE Transactions on Knowledge and Data Engineering


Video-EM: Event-Centric Episodic Memory for Long-Form Video Understanding

Abstract: Video Large Language Models (Video-LLMs) have shown strong video understanding, yet their application to long-form videos remains constrained by limited context windows. A common workaround is to compress long videos into a handful of representative frames via retrieval or summarization. However, most existing pipelines score frames in isolation, implicitly assuming that frame-level saliency is sufficient for downstream reasoning. This often yields redundant selections, fragmented temporal evidence, and weakened narrative grounding for long-form video question answering. We present \textbf{Video-EM}, a training-free, event-centric episodic memory framework that reframes long-form VideoQA as \emph{episodic event construction} followed by \emph{memory refinement}. Instead of treating retrieved keyframes as independent visuals, Video-EM employs an LLM as an active memory agent to orchestrate off-the-shelf tools: it first localizes query-relevant moments via multi-grained semantic matching, then groups and segments them into temporally coherent events, and finally encodes each event as a grounded episodic memory with explicit temporal indices and spatio-temporal cues (capturing \emph{when}, \emph{where}, \emph{what}, and involved entities). To further suppress verbosity and noise from imperfect upstream signals, Video-EM integrates a reasoning-driven self-reflection loop that iteratively verifies evidence sufficiency and cross-event consistency, removes redundancy, and adaptively adjusts event granularity. The outcome is a compact yet reliable \emph{event timeline} -- a minimal but sufficient episodic memory set that can be directly consumed by existing Video-LLMs without additional training or architectural changes.

Yun Wang
Long Zhang
Jingren Liu
Jiaqi Yan
Zhanjie Zhang
Jiahao Zheng
Ao Ma
Run Ling
Xun Yang
Dapeng Wu
Xiangyu Chen
Xuelong Li


Beyond Naïve Prompting: Strategies for Improved Context-aided Forecasting with LLMs

Abstract: Real-world forecasting requires models to integrate not only historical data but also relevant contextual information provided in textual form. While large language models (LLMs) show promise for context-aided forecasting, critical challenges remain: we lack diagnostic tools to understand failure modes, performance remains far below their potential, and high computational costs limit practical deployment. We introduce a unified framework of four strategies that address these limitations along three orthogonal dimensions: model diagnostics, accuracy, and efficiency. Through extensive evaluation across model families from small open-source models to frontier models including Gemini, GPT, and Claude, we uncover both fundamental insights and practical solutions. Our findings span three key dimensions: diagnostic strategies reveal the "Execution Gap" where models correctly explain how context affects forecasts but fail to apply this reasoning; accuracy-focused strategies achieve substantial performance improvements of 25-50%; and efficiency-oriented approaches show that adaptive routing between small and large models can approach large model accuracy on average while significantly reducing inference costs. These orthogonal strategies can be flexibly integrated based on deployment constraints, providing practitioners with a comprehensive toolkit for practical LLM-based context-aided forecasting.

Arjun Ashok
Andrew Robert Williams
Vincent Zhihao Zheng
Irina Rish
Nicolas Chapados
Étienne Marcotte
Valentina Zantedeschi
Alexandre Drouin


Architecting Long-Context LLM Acceleration with Packing-Prefetch Scheduler and Ultra-Large Capacity On-Chip Memories

Abstract: Long-context Large Language Model (LLM) inference faces increasing compute bottlenecks as attention calculations scale with context length, primarily due to the growing KV-cache transfer overhead that saturates High Bandwidth Memory (HBM). While prefetching techniques mitigate cache misses by fetching KV data in advance, their spatial and temporal benefits present new opportunities to exploit. This work proposes a packing-prefetch scheduling architecture with monolithic 3D (M3D) back-end-of-line (BEOL) compatible embedded memories with ultra-large on-chip capacity to accelerate long-context LLM inference. Our optimizations demonstrate 8.06x decode speedup and 1.83x overall latency reduction on Llama3.1-8B using TPUv6e-like hardware with additional 512MB BEOL memories over the serial execution. Evaluations of multi-request workloads on TPU-like architectures show 1.7x-2.4x throughput improvement and 1.5x-2.4x HBM bandwidth reduction compared to packing-only methods on Llama3.1-8B and Llama3.1-70B models. With the co-design of packing, prefetching, and BEOL memories, our approach alleviates HBM constraints and enables efficient long-context LLM inference.

Ming-Yen Lee
Faaiq Waqar
Hanchen Yang
Muhammed Ahosan Ul Karim
Harsono Simka
Shimeng Yu


Positional Biases Shift as Inputs Approach Context Window Limits

Abstract: Large Language Models (LLMs) often struggle to use information across long inputs effectively. Prior work has identified positional biases, such as the Lost in the Middle (LiM) effect, where models perform better when information appears at the beginning (primacy bias) or end (recency bias) of the input, rather than in the middle. However, long-context studies have not consistently replicated these effects, raising questions about their intensity and the conditions under which they manifest. To address this, we conducted a comprehensive analysis using relative rather than absolute input lengths, defined with respect to each model's context window. Our findings reveal that the LiM effect is strongest when inputs occupy up to 50% of a model's context window. Beyond that, the primacy bias weakens, while recency bias remains relatively stable. This effectively eliminates the LiM effect; instead, we observe a distance-based bias, where model performance is better when relevant information is closer to the end of the input. Furthermore, our results suggest that successful retrieval is a prerequisite for reasoning in LLMs, and that the observed positional biases in reasoning are largely inherited from retrieval. These insights have implications for long-context tasks, the design of future LLM benchmarks, and evaluation methodologies for LLMs handling extended inputs.

Blerta Veseli
Julian Chibane
Mariya Toneva
Alexander Koller


Context Engineering for Multi-Agent LLM Code Assistants Using Elicit, NotebookLM, ChatGPT, and Claude Code

Abstract: Large Language Models (LLMs) have shown promise in automating code generation and software engineering tasks, yet they often struggle with complex, multi-file projects due to context limitations and knowledge gaps. We propose a novel context engineering workflow that combines multiple AI components: an Intent Translator (GPT-5) for clarifying user requirements, an Elicit-powered semantic literature retrieval for injecting domain knowledge, NotebookLM-based document synthesis for contextual understanding, and a Claude Code multi-agent system for code generation and validation. Our integrated approach leverages intent clarification, retrieval-augmented generation, and specialized sub-agents orchestrated via Claude's agent framework. We demonstrate that this method significantly improves the accuracy and reliability of code assistants in real-world repositories, yielding higher single-shot success rates and better adherence to project context than baseline single-agent approaches. Qualitative results on a large Next.js codebase show the multi-agent system effectively plans, edits, and tests complex features with minimal human intervention. We compare our system with recent frameworks like CodePlan, MASAI, and HyperAgent, highlighting how targeted context injection and agent role decomposition lead to state-of-the-art performance. Finally, we discuss the implications for deploying LLM-based coding assistants in production, along with lessons learned on context management and future research directions.


SlimInfer: Accelerating Long-Context LLM Inference via Dynamic Token Pruning

Abstract: Long-context inference for Large Language Models (LLMs) is heavily limited by high computational demands. While several existing methods optimize attention computation, they still process the full set of hidden states at each layer, limiting overall efficiency. In this work, we propose SlimInfer, an innovative framework that aims to accelerate inference by directly pruning less critical prompt tokens during the forward pass. Our key insight is an information diffusion phenomenon: As information from critical tokens propagates through layers, it becomes distributed across the entire sequence. This diffusion process suggests that LLMs can maintain their semantic integrity when excessive tokens, even including these critical ones, are pruned in hidden states. Motivated by this, SlimInfer introduces a dynamic fine-grained pruning mechanism that accurately removes redundant tokens of hidden state at intermediate layers. This layer-wise pruning naturally enables an asynchronous KV cache manager that prefetches required token blocks without complex predictors, reducing both memory usage and I/O costs. Extensive experiments show that SlimInfer can achieve up to $\mathbf{2.53\times}$ time-to-first-token (TTFT) speedup and $\mathbf{1.88\times}$ end-to-end latency reduction for LLaMA3.1-8B-Instruct on a single RTX 4090, without sacrificing performance on LongBench. Our code is available at https://github.com/Longxmas/SlimInfer.

Lingkun Long
Rubing Yang
Yushi Huang
Desheng Hui
Ao Zhou
Jianlei Yang


Sculptor: Empowering LLMs with Cognitive Agency via Active Context Management

Abstract: Large Language Models (LLMs) suffer from significant performance degradation when processing long contexts due to proactive interference, where irrelevant information in earlier parts of the context disrupts reasoning and memory recall. While most research focuses on external memory systems to augment LLMs' capabilities, we propose a complementary approach: empowering LLMs with Active Context Management (ACM) tools to actively sculpt their internal working memory. We introduce Sculptor, a framework that equips LLMs with three categories of tools: (1) context fragmentation, (2) summary, hide, and restore, and (3) precise search. Our approach enables LLMs to proactively manage their attention and working memory, analogous to how humans selectively focus on relevant information while filtering out distractions. Experimental evaluation on diverse long-context benchmarks demonstrates that Sculptor significantly improves performance even without specific training, leveraging LLMs' inherent tool-calling and instruction-following capabilities. To further optimize these strategies, we introduce a novel dynamic context-aware reinforcement learning (RL) approach, advancing the training of an agent that actively modifies its own conversational history. By enabling Active Context Management, Sculptor not only mitigates proactive interference but also provides a cognitive foundation for more reliable reasoning across diverse long-context tasks-highlighting that explicit context-control strategies, rather than merely larger token windows, are key to robustness at scale.

Mo Li
L. H. Xu
Qitai Tan
Long Ma
Ting Cao
Yunxin Liu


LaMPE: Length-aware Multi-grained Positional Encoding for Adaptive Long-context Scaling Without Training

Abstract: Large language models (LLMs) experience significant performance degradation when the input exceeds the pretraining context window, primarily due to the out-of-distribution (OOD) behavior of Rotary Position Embedding (RoPE). Recent studies mitigate this problem by remapping OOD positions into the in-distribution range with fixed mapping strategies, ignoring the dynamic relationship between input length and the model's effective context window. To this end, we propose Length-aware Multi-grained Positional Encoding (LaMPE), a training-free method that fully utilizes the model's effective context window for adaptive long-context scaling in LLMs. Motivated by the left-skewed frequency distribution of relative positions, LaMPE establishes a dynamic relationship between mapping length and input length through a parametric scaled sigmoid function to adaptively allocate positional capacity across varying input lengths. Meanwhile, LaMPE devises a novel multi-grained attention mechanism that strategically allocates positional resolution across different sequence regions to capture both fine-grained locality and long-range dependencies. Our method can be seamlessly applied to a wide range of RoPE-based LLMs without training. Extensive experiments on three representative LLMs across five mainstream long-context benchmarks demonstrate that LaMPE achieves significant performance improvements compared to existing length extrapolation methods. The code will be released at https://github.com/scar-on/LaMPE.

Sikui Zhang
Guangze Gao
Ziyun Gan
Chunfeng Yuan
Zefeng Lin
Houwen Peng
Bing Li
Weiming Hu


KCR: Resolving Long-Context Knowledge Conflicts via Reasoning in LLMs

Abstract: Knowledge conflicts commonly arise across diverse sources, and their prevalence has increased with the advent of LLMs. When dealing with conflicts between multiple contexts, also known as \emph{inter-context knowledge conflicts}, LLMs are often confused by lengthy and conflicting contexts. To address this challenge, we propose the Knowledge Conflict Reasoning (KCR) framework, which enhances the ability of LLMs to resolve conflicting knowledge. The key idea of KCR is to train backbone LLMs to establish a correct reasoning process by rewarding them for selecting and adhering to the context with stronger logical consistency when presented with conflicting contexts. Specifically, we first extract reasoning paths, represented by either text or local knowledge graphs, from the conflicting long contexts. Subsequently, we employ Reinforcement Learning to encourage the model to learn the paradigm of reasoning process that follows correct reasoning paths rather than the incorrect counterparts. This enables the backbone models to genuinely acquire the capability to resolve inter-context knowledge conflicts within long contexts. Experimental results demonstrate that our framework significantly improves the ability of various backbone models to resolve knowledge conflicts in long-context scenarios, yielding substantial performance gains.

Xianda Zheng
Zijian Huang
Meng-Fen Chiang
Michael J. Witbrock
Kaiqi Zhao


TextQuests: How Good are LLMs at Text-Based Video Games?

Abstract: Evaluating AI agents within complex, interactive environments that mirror real-world challenges is critical for understanding their practical capabilities. While existing agent benchmarks effectively assess skills like tool use or performance on structured tasks, they often do not fully capture an agent's ability to operate autonomously in exploratory environments that demand sustained, self-directed reasoning over a long and growing context. To enable a more accurate assessment of AI agents in challenging exploratory environments, we introduce TextQuests, a benchmark based on the Infocom suite of interactive fiction games. These text-based adventures, which can take human players over 30 hours and require hundreds of precise actions to solve, serve as an effective proxy for evaluating AI agents on focused, stateful tasks. The benchmark is specifically designed to assess an LLM agent's capacity for self-contained problem-solving by precluding the use of external tools, thereby focusing on intrinsic long-context reasoning capabilities in an exploratory environment characterized by the need for trial-and-error learning and sustained problem-solving within a single interactive session. We release TextQuests at https://textquests.ai.

Long Phan
Mantas Mazeika
Andy Zou
Dan Hendrycks


Context-aware Rotary Position Embedding

Abstract: Positional encoding is a vital component of Transformer architectures, enabling models to incorporate sequence order into self-attention mechanisms. Rotary Positional Embeddings (RoPE) have become a widely adopted solution due to their compatibility with relative position encoding and computational efficiency. However, RoPE relies on static, input-independent sinusoidal frequency patterns, limiting its ability to model context-sensitive relationships. In this work, we propose CARoPE (Context-Aware Rotary Positional Embedding), a novel generalization of RoPE that dynamically generates head-specific frequency patterns conditioned on token embeddings. This design introduces token- and context-sensitive positional representations while preserving RoPE efficiency and architectural simplicity. CARoPE computes input-dependent phase shifts using a bounded transformation of token embeddings and integrates them into the rotary mechanism across attention heads. We evaluate CARoPE on the FineWeb-Edu-10B dataset using GPT-2 variants trained on next-token prediction tasks. Experimental results show that CARoPE consistently outperforms RoPE and other common positional encoding baselines, achieving significantly lower perplexity, even at longer context lengths. Additionally, CARoPE enables faster training throughput without sacrificing model stability. These findings demonstrate that CARoPE offers a scalable, expressive, and efficient upgrade to existing positional encoding strategies in Transformer models.

Ali Veisi
Delaram Fartoot
Hamidreza Amirzadeh


MCP4EDA: LLM-Powered Model Context Protocol RTL-to-GDSII Automation with Backend Aware Synthesis Optimization

Abstract: This paper presents MCP4EDA, the first Model Context Protocol server that enables Large Language Models (LLMs) to control and optimize the complete open-source RTL-to-GDSII design flow through natural language interaction. The system integrates Yosys synthesis, Icarus Verilog simulation, OpenLane place-and-route, GTKWave analysis, and KLayout visualization into a unified LLM-accessible interface, enabling designers to execute complex multi-tool EDA workflows conversationally via AI assistants such as Claude Desktop and Cursor IDE. The principal contribution is a backend-aware synthesis optimization methodology wherein LLMs analyze actual post-layout timing, power, and area metrics from OpenLane results to iteratively refine synthesis TCL scripts, establishing a closed-loop optimization system that bridges the traditional gap between synthesis estimates and physical implementation reality. In contrast to conventional flows that rely on wire-load models, this methodology leverages real backend performance data to guide synthesis parameter tuning, optimization sequence selection, and constraint refinement, with the LLM functioning as an intelligent design space exploration agent. Experimental evaluation on representative digital designs demonstrates 15-30% improvements in timing closure and 10-20% area reduction compared to default synthesis flows, establishing MCP4EDA as the first practical LLM-controlled end-to-end open-source EDA automation system. The code and demo are avaiable at: http://www.agent4eda.com/

Yiting Wang
Wanghao Ye
Yexiao He
Yiran Chen
Gang Qu
Ang Li


Smooth Reading: Bridging the Gap of Recurrent LLM to Self-Attention LLM on Long-Context Tasks

Abstract: Recently, recurrent large language models (Recurrent LLMs) with linear computational complexity have re-emerged as efficient alternatives to self-attention-based LLMs (Self-Attention LLMs), which have quadratic complexity. However, Recurrent LLMs often underperform on long-context tasks due to their limited fixed-size memory. Previous research has primarily focused on enhancing the memory capacity of Recurrent LLMs through architectural innovations, but these approaches have not yet enabled Recurrent LLMs to match the performance of Self-Attention LLMs on long-context tasks. We argue that this limitation arises because processing the entire context at once is not well-suited for Recurrent LLMs. In this paper, we propose Smooth Reading, a chunk-wise inference method inspired by human reading strategies. Smooth Reading processes context in chunks and iteratively summarizes the contextual information, thereby reducing memory demands and making the approach more compatible with Recurrent LLMs. Our experimental results show that this method substantially narrows the performance gap between Recurrent and Self-Attention LLMs on long-context tasks, while preserving the efficiency advantages of Recurrent LLMs. Our Smooth Reading boosts SWA-3B-4k (a Recurrent LLM) from 5.68% lower to 3.61% higher performance than Self-Attention LLMs on LongBench. Besides, our method maintains the high efficiency, training 3x faster and inferring 2x faster at 64k context compared to Self-Attention LLMs. To our knowledge, this is the first work to achieve comparable performance using Recurrent LLMs compared with Self-Attention LLMs on long-context tasks. We hope our method will inspire future research in this area. To facilitate further progress, we will release code and dataset.

Kai Liu
Zhan Su
Peijie Dong
Fengran Mo
Jianfei Gao
ShaoTing Zhang
Kai Chen


Ref-Long: Benchmarking the Long-context Referencing Capability of Long-context Language Models

Abstract: Long-context language models (LCLMs) have exhibited impressive capabilities in long-context understanding tasks. Among these, long-context referencing -- a crucial task that requires LCLMs to attribute items of interest to specific parts of long-context data -- remains underexplored. To bridge this gap, this paper proposes Referencing Evaluation for Long-context Language Models (Ref-Long), a novel benchmark designed to assess the long-context referencing capability of LCLMs. Specifically, Ref-Long requires LCLMs to identify the indexes of documents that reference a specific key, emphasizing contextual relationships between the key and the documents over simple retrieval. Based on the task design, we construct three subsets ranging from synthetic to realistic scenarios to form the Ref-Long benchmark. Experimental results of 13 LCLMs reveal significant shortcomings in long-context referencing, even among advanced models like GPT-4o. To further investigate these challenges, we conduct comprehensive analyses, including human evaluations, task format adjustments, fine-tuning experiments, and error analyses, leading to several key insights. Our data and code can be found in https://github. com/wujunjie1998/Ref-Long.

Junjie Wu
Gefei Gu
Yanan Zheng
Dit-Yan Yeung
Arman Cohan

Annual Meeting of the Association for Computational Linguistics


ETT: Expanding the Long Context Understanding Capability of LLMs at Test-Time

Abstract: Transformer-based Language Models' computation and memory overhead increase quadratically as a function of sequence length. The quadratic cost poses challenges when employing LLMs for processing long sequences. In this work, we introduce \ourmodelacronym~(Extend at Test-Time), method for extending the context length of short context Transformer-based LLMs, with constant memory requirement and linear computation overhead. ETT enable the extension of the context length at test-time by efficient fine-tuning the model's parameters on the input context, chunked into overlapping small subsequences. We evaluate ETT on LongBench by extending the context length of GPT-Large and Phi-2 up to 32 times, increasing from 1k to 32k tokens. This results in up to a 30 percent improvement in the model's accuracy. We also study how context can be stored in LLM's weights effectively and efficiently. Through a detailed ablation study, we examine which Transformer modules are most beneficial to fine-tune at test-time. Interestingly, we find that fine-tuning the second layer of the FFNs is more effective than full fine-tuning, leading to a further improvement in the models' accuracy.

Kiarash Zahirnia
Zahra Golpayegani
Walid Ahmed
Yang Liu


PERK: Long-Context Reasoning as Parameter-Efficient Test-Time Learning

Abstract: Long-context reasoning requires accurately identifying relevant information in extensive, noisy input contexts. Previous research shows that using test-time learning to encode context directly into model parameters can effectively enable reasoning over noisy information. However, meta-learning methods for enabling test-time learning are prohibitively memory-intensive, preventing their application to long context settings. In this work, we propose PERK (Parameter Efficient Reasoning over Knowledge), a scalable approach for learning to encode long input contexts using gradient updates to a lightweight model adapter at test time. Specifically, PERK employs two nested optimization loops in a meta-training phase. The inner loop rapidly encodes contexts into a low-rank adapter (LoRA) that serves as a parameter-efficient memory module for the base model. Concurrently, the outer loop learns to use the updated adapter to accurately recall and reason over relevant information from the encoded long context. Our evaluations on several long-context reasoning tasks show that PERK significantly outperforms the standard prompt-based long-context baseline, achieving average absolute performance gains of up to 90% for smaller models (GPT-2) and up to 27% for our largest evaluated model, Qwen-2.5-0.5B. In general, PERK is more robust to reasoning complexity, length extrapolation, and the locations of relevant information in contexts. Finally, we show that while PERK is memory-intensive during training, it scales more efficiently at inference time than prompt-based long-context inference.

Zeming Chen
Angelika Romanou
Gail Weiss
Antoine Bosselut


Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities

Abstract: In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our most capable model yet, achieving SoTA performance on frontier coding and reasoning benchmarks. In addition to its incredible coding and reasoning skills, Gemini 2.5 Pro is a thinking model that excels at multimodal understanding and it is now able to process up to 3 hours of video content. Its unique combination of long context, multimodal and reasoning capabilities can be combined to unlock new agentic workflows. Gemini 2.5 Flash provides excellent reasoning abilities at a fraction of the compute and latency requirements and Gemini 2.0 Flash and Flash-Lite provide high performance at low latency and cost. Taken together, the Gemini 2.X model generation spans the full Pareto frontier of model capability vs cost, allowing users to explore the boundaries of what is possible with complex agentic problem solving.

Gheorghe Comanici
Eric Bieber
Mike Schaekermann
Ice Pasupat
Noveen Sachdeva
Inderjit Dhillon
Marcel Blistein
Ori Ram
Dan Zhang
Evan Rosen
Luke Marris
Sam Petulla
Colin Gaffney
Asaf Aharoni
Nathan Lintz
Tiago Cardal Pais
Henrik Jacobsson
Idan Szpektor
Nan-Jiang Jiang
Krishna Haridasan
Ahmed Omran
Nikunj Saunshi
Dara Bahri
Gaurav Mishra
Eric Chu
Toby Boyd
Brad Hekman
Aaron Parisi
Chaoyi Zhang
Kornraphop Kawintiranon
Tania Bedrax-Weiss
Oliver Wang
Ya Xu
Ollie Purkiss
Uri Mendlovic
Ilaï Deutel
Nam Nguyen
Adam Langley
Flip Korn
Lucia Rossazza
Alexandre Ramé
Sagar Waghmare
Helen Miller
Nathan Byrd
Ashrith Sheshan
Raia Hadsell
Sangnie Bhardwaj
Pawel Janus
Tero Rissa
Dan Horgan
Alvin Abdagic
Lior Belenki
James Allingham
Anima Singh
Theo Guidroz
Srivatsan Srinivasan
Herman Schmit
Kristen Chiafullo
Andre Elisseeff
Nilpa Jha
Prateek Kolhar
Leonard Berrada
Frank Ding
Xiance Si
Shrestha Basu Mallick
Franz Och
Sofia Erell
Eric Ni
Tejasi Latkar
Sherry Yang
Petar Sirkovic
Ziqiang Feng
Robert Leland
Rachel Hornung
Gang Wu
Charles Blundell
Hamidreza Alvari
Po-Sen Huang
Cathy Yip
Sanja Deur
Li Liu
Gabriela Surita
Pablo Duque
Dima Damen
Johnson Jia
Arthur Guez
Markus Mircea
Animesh Sinha
Alberto Magni
Paweł Stradomski
Tal Marian
Vlado Galić
Wenhu Chen
Hisham Husain
Achintya Singhal
Dominik Grewe
François-Xavier Aubet
Shuang Song
Lorenzo Blanco
Leland Rechis
Lewis Ho
Rich Munoz
Kelvin Zheng
Jessica Hamrick
Kevin Mather
Hagai Taitelbaum
Eliza Rutherford
Yun Lei
Kuangyuan Chen
Anand Shukla
Erica Moreira
Eric Doi
Berivan Isik
Nir Shabat
Dominika Rogozińska
Kashyap Kolipaka
Jason Chang
Eugen Vušak
Srinivasan Venkatachary
Shadi Noghabi
Tarun Bharti
Younghoon Jun
Aleksandr Zaks
Simon Green
Jeshwanth Challagundla
William Wong
Muqthar Mohammad
Dean Hirsch
Yong Cheng
Iftekhar Naim
Lev Proleev
Damien Vincent
Aayush Singh
Maxim Krikun
Dilip Krishnan
Zoubin Ghahramani
Aviel Atias
Rajeev Aggarwal
Christo Kirov
Dimitrios Vytiniotis
Christy Koh
Alexandra Chronopoulou
Pawan Dogra
Vlad-Doru Ion
Gladys Tyen
Jason Lee
Felix Weissenberger
Trevor Strohman
Ashwin Balakrishna
Jack Rae
Marko Velic
Raoul de Liedekerke
Oded Elyada
Wentao Yuan
Canoee Liu
Lior Shani
Sergey Kishchenko
Bea Alessio
Yandong Li
Richard Song
Sam Kwei
Orion Jankowski
Aneesh Pappu
Youhei Namiki
Yenai Ma
Nilesh Tripuraneni
Colin Cherry
Marissa Ikonomidis
Yu-Cheng Ling
Colin Ji
Beka Westberg
Auriel Wright
Da Yu
David Parkinson
Swaroop Ramaswamy
Jerome Connor
Soheil Hassas Yeganeh
Snchit Grover
George Kenwright
Lubo Litchev
Chris Apps
Alex Tomala
Felix Halim
Alex Castro-Ros
Zefei Li
Anudhyan Boral
Pauline Sho
Michal Yarom
Eric Malmi
David Klinghoffer
Rebecca Lin
Alan Ansell
Pradeep Kumar S
Shubin Zhao
Siqi Zuo
Adam Santoro
Heng-Tze Cheng
Solomon Demmessie
Yuchi Liu
Nicole Brichtova
Allie Culp
Nathaniel Braun
Dan Graur
Will Ng
Nikhil Mehta
Aaron Phillips
Patrik Sundberg
Varun Godbole
Fangyu Liu
Yash Katariya
David Rim
Mojtaba Seyedhosseini
Sean Ammirati
Jonas Valfridsson
Mahan Malihi
Timothy Knight
Andeep Toor
Thomas Lampe
Abe Ittycheriah
Lewis Chiang
Chak Yeung
Alexandre Fréchette
Jinmeng Rao
Huisheng Wang
Himanshu Srivastava
Richard Zhang
Rocky Rhodes
Ariel Brand
Dean Weesner
Ilya Figotin
Felix Gimeno
Rachana Fellinger
Pierre Marcenac
José Leal
Eyal Marcus
Victor Cotruta
Rodrigo Cabrera
Sheryl Luo
Dan Garrette
Vera Axelrod
Sorin Baltateanu
David Barker
Dongkai Chen
Horia Toma
Ben Ingram
Jason Riesa
Chinmay Kulkarni
Yujing Zhang
Hongbin Liu
Chao Wang
Martin Polacek
Will Wu
Kai Hui
Adrian N Reyes
Yi Su
Megan Barnes
Ishaan Malhi
Anfal Siddiqui
Qixuan Feng
Mihai Damaschin
Daniele Pighin
Andreas Steiner
Samuel Yang
Ramya Sree Boppana
Simeon Ivanov
Arun Kandoor
Aditya Shah
Asier Mujika
Da Huang
Christopher A. Choquette-Choo
Mohak Patel
Tianhe Yu
Toni Creswell
Jerry
Liu
Catarina Barros
Yasaman Razeghi
Aurko Roy
Phil Culliton
Binbin Xiong
Jiaqi Pan
Thomas Strohmann
Tolly Powell
Babi Seal
Doug DeCarlo
Pranav Shyam
Kaan Katircioglu
Xuezhi Wang
Cassidy Hardin
Immanuel Odisho
Josef Broder
Oscar Chang
Arun Nair
Artem Shtefan
Maura O'Brien
Manu Agarwal
Sahitya Potluri
Siddharth Goyal
Amit Jhindal
Saksham Thakur
Yury Stuken
James Lyon
Kristina Toutanova
Fangxiaoyu Feng
Austin Wu
Ben Horn
Alek Wang
Alex Cullum
Gabe Taubman
Disha Shrivastava
Chongyang Shi
Hamish Tomlinson
Roma Patel
Tao Tu
Ada Maksutaj Oflazer
Francesco Pongetti
Mingyao Yang
Adrien Ali Taïga
Vincent Perot
Nuo Wang Pierse
Feng Han
Yoel Drori
Iñaki Iturrate
Ayan Chakrabarti
Legg Yeung
Dave Dopson
Yi-ting Chen
Apoorv Kulshreshtha
Tongfei Guo
Philip Pham
Tal Schuster
Junquan Chen
Alex Polozov
Jinwei Xing
Huanjie Zhou
Praneeth Kacham
Doron Kukliansky
Antoine Miech
Sergey Yaroshenko
Ed Chi
Sholto Douglas
Hongliang Fei
Mathieu Blondel
Preethi Myla
Lior Madmoni
Xing Wu
Daniel Keysers
Kristian Kjems
Isabela Albuquerque
Lijun Yu
Joel D'sa
Michelle Plantan
Vlad Ionescu
Jaume Sanchez Elias
Abhirut Gupta
Manish Reddy Vuyyuru
Fred Alcober
Tong Zhou
Kaiyang Ji
Florian Hartmann
Subha Puttagunta
Hugo Song
Ehsan Amid
Anca Stefanoiu
Andrew Lee
Paul Pucciarelli
Emma Wang
Amit Raul
Slav Petrov
Isaac Tian
Valentin Anklin
Nana Nti
Victor Gomes
Max Schumacher
Grace Vesom
Alex Panagopoulos
Konstantinos Bousmalis
Daniel Andor
Josh Jacob
Yuan Zhang
Bill Rosgen
Matija Kecman
Matthew Tung
Alexandra Belias
Noah Goodman
Paul Covington
Brian Wieder
Nikita Saxena
Elnaz Davoodi
Muhuan Huang
Sharath Maddineni
Vincent Roulet
Folawiyo Campbell-Ajala
Pier Giuseppe Sessa
Xintian
Wu
Guangda Lai
Paul Collins
Alex Haig
Vytenis Sakenas
Xiaowei Xu
Marissa Giustina
Laurent El Shafey
Pichi Charoenpanit
Shefali Garg
Joshua Ainslie
Boone Severson
Montse Gonzalez Arenas
Shreya Pathak
Sujee Rajayogam
Jie Feng
Michiel Bakker
Sheng Li
Nevan Wichers
Jamie Rogers
Xinyang Geng
Yeqing Li
Rolf Jagerman
Chao Jia
Nadav Olmert
David Sharon
Matthew Mauger
Sandeep Mariserla
Hongxu Ma
Megha Mohabey
Kyuyeun Kim
Alek Andreev
Scott Pollom
Juliette Love
Vihan Jain
Priyanka Agrawal
Yannick Schroecker
Alisa Fortin
Manfred Warmuth
Ji Liu
Andrew Leach
Irina Blok
Ganesh Poomal Girirajan
Roee Aharoni
Benigno Uria
Andrei Sozanschi
Dan Goldberg
Lucian Ionita
Marco Tulio Ribeiro
Martin Zlocha
Vighnesh Birodkar
Sami Lachgar
Liangzhe Yuan
Himadri Choudhury
Matt Ginsberg
Fei Zheng
Gregory Dibb
Emily Graves
Swachhand Lokhande
Gabriel Rasskin
George-Cristian Muraru
Corbin Quick
Sandeep Tata
Pierre Sermanet
Aditya Chawla
Itay Karo
Yan Wang
Susan Zhang
Orgad Keller
Anca Dragan
Guolong Su
Ian Chou
Xi Liu
Yiqing Tao
Shruthi Prabhakara
Marc Wilson
Ruibo Liu
Shibo Wang
Georgie Evans
David Du
Alfonso Castaño
Gautam Prasad
Mona El Mahdy
Sebastian Gerlach
Machel Reid
Jarrod Kahn
Amir Zait
Thanumalayan Sankaranarayana Pillai
Thatcher Ulrich
Guanyu Wang
Jan Wassenberg
Efrat Farkash
Kiran Yalasangi
Congchao Wang
Maria Bauza
Simon Bucher
Ting Liu
Jun Yan
Gary Leung
Vikas Sindhwani
Parker Barnes
Avi Singh
Ivan Jurin
Jichuan Chang
Niket Kumar Bhumihar
Sivan Eiger
Gui Citovsky
Ben Withbroe
Zhang Li
Siyang Xue
Niccolò Dal Santo
Georgi Stoyanov
Yves Raimond
Steven Zheng
Yilin Gao
Vít Listík
Sławek Kwasiborski
Rachel Saputro
Adnan Ozturel
Ganesh Mallya
Kushal Majmundar
Ross West
Paul Caron
Jinliang Wei
Lluis Castrejon
Sharad Vikram
Deepak Ramachandran
Nikhil Dhawan
Jiho Park
Sara Smoot
George van den Driessche
Yochai Blau
Chase Malik
Wei Liang
Roy Hirsch
Cicero Nogueira dos Santos
Eugene Weinstein
Aäron van den Oord
Sid Lall
Nicholas FitzGerald
Zixuan Jiang
Xuan Yang
Dale Webster
Ali Elqursh
Aedan Pope
Georges Rotival
David Raposo
Wanzheng Zhu
Jeff Dean
Sami Alabed
Dustin Tran
Arushi Gupta
Zach Gleicher
Jessica Austin
Edouard Rosseel
Megh Umekar
Dipanjan Das
Yinghao Sun
Kai Chen
Karolis Misiunas
Xiang Zhou
Yixian Di
Alyssa Loo
Josh Newlan
Bo Li
Vinay Ramasesh
Ying Xu
Alex Chen
Sudeep Gandhe
Radu Soricut
Nikita Gupta
Shuguang Hu
Seliem El-Sayed
Xavier Garcia
Idan Brusilovsky
Pu-Chin Chen
Andrew Bolt
Lu Huang
Alex Gurney
Zhiying Zhang
Alexander Pritzel
Jarek Wilkiewicz
Bryan Seybold
Bhargav Kanagal Shamanna
Felix Fischer
Josef Dean
Karan Gill
Ross Mcilroy
Abhishek Bhowmick
Jeremy Selier
Antoine Yang
Derek Cheng
Vladimir Magay
Jie Tan
Dhriti Varma
Christian Walder
Tomas Kocisky
Ryo Nakashima
Paul Natsev
Mike Kwong
Ionel Gog
Chiyuan Zhang
Sander Dieleman
Thomas Jimma
Andrey Ryabtsev
Siddhartha Brahma
David Steiner
Dayou Du
Ante Žužul
Mislav Žanić
Mukund Raghavachari
Willi Gierke
Zeyu Zheng
Dessie Petrova
Yann Dauphin
Yuchuan Liu
Ido Kessler
Steven Hand
Chris Duvarney
Seokhwan Kim
Hyo Lee
Léonard Hussenot
Jeffrey Hui
Josh Smith
Deepali Jain
Jiawei Xia
Gaurav Singh Tomar
Keyvan Amiri
Du Phan
Fabian Fuchs
Tobias Weyand
Nenad Tomasev
Alexandra Cordell
Xin Liu
Jonathan Mallinson
Pankaj Joshi
Andy Crawford
Arun Suggala
Steve Chien
Nick Fernando
Mariella Sanchez-Vargas
Duncan Williams
Phil Crone
Xiyang Luo
Igor Karpov
Jyn Shan
Terry Thurk
Robin Strudel
Paul Voigtlaender
Piyush Patil
Tim Dozat
Ali Khodaei
Sahil Singla
Piotr Ambroszczyk
Qiyin Wu
Yifan Chang
Brian Roark
Chaitra Hegde
Tianli Ding
Angelos Filos
Zhongru Wu
André Susano Pinto
Shuang Liu
Saarthak Khanna
Aditya Pandey
Siobhan Mcloughlin
Qiujia Li
Sam Haves
Allan Zhou
Elena Buchatskaya
Isabel Leal
Peter de Boursac
Nami Akazawa
Nina Anderson
Terry Chen
Krishna Somandepalli
Chen Liang
Sheela Goenka
Stephanie Winkler
Alexander Grushetsky
Yifan Ding
Jamie Smith
Fan Ye
Jordi Pont-Tuset
Eric Li
Ruichao Li
Tomer Golany
Dawid Wegner
Tao Jiang
Omer Barak
Yuan Shangguan
Eszter Vértes
Renee Wong
Jörg Bornschein
Alex Tudor
Michele Bevilacqua
Tom Schaul
Ankit Singh Rawat
Yang Zhao
Kyriakos Axiotis
Lei Meng
Cory McLean
Jonathan Lai
Jennifer Beattie
Nate Kushman
Yaxin Liu
Blair Kutzman
Fiona Lang
Jingchen Ye
Praneeth Netrapalli
Pushkar Mishra
Myriam Khan
Megha Goel
Rob Willoughby
David Tian
Honglei Zhuang
JD Chen
Zak Tsai
Tasos Kementsietsidis
Arjun Khare
James Keeling
Keyang Xu
Nathan Waters
Florent Altché
Ashok Popat
Bhavishya Mittal
David Saxton
Dalia El Badawy
Michael Mathieu
Zheng Zheng
Hao Zhou
Nishant Ranka
Richard Shin
Qingnan Duan
Tim Salimans
Ioana Mihailescu
Uri Shaham
Ming-Wei Chang
Yannis Assael
Nishanth Dikkala
Martin Izzard
Vincent Cohen-Addad
Cat Graves
Vlad Feinberg
Grace Chung
DJ Strouse
Danny Karmon
Sahand Sharifzadeh
Zoe Ashwood
Khiem Pham
Jon Blanton
Alex Vasiloff
Jarred Barber
Mark Geller
Aurick Zhou
Fedir Zubach
Tzu-Kuo Huang
Lei Zhang
Himanshu Gupta
Matt Young
Julia Proskurnia
Ronny Votel
Valentin Gabeur
Gabriel Barcik
Aditya Tripathi
Hongkun Yu
Geng Yan
Beer Changpinyo
Filip Pavetić
Amy Coyle
Yasuhisa Fujii
Jorge Gonzalez Mendez
Tianhao Zhou
Harish Rajamani
Blake Hechtman
Eddie Cao
Da-Cheng Juan
Yi-Xuan Tan
Valentin Dalibard
Yilun Du
Natalie Clay
Kaisheng Yao
Wenhao Jia
Dimple Vijaykumar
Yuxiang Zhou
Xinyi Bai
Wei-Chih Hung
Steven Pecht
Georgi Todorov
Nikhil Khadke
Pramod Gupta
Preethi Lahoti
Arnaud Autef
Karthik Duddu
James Lee-Thorp
Alexander Bykovsky
Tautvydas Misiunas
Sebastian Flennerhag
Santhosh Thangaraj
Jed McGiffin
Zack Nado
Markus Kunesch
Andreas Noever
Amir Hertz
Marco Liang
Victor Stone
Evan Palmer
Samira Daruki
Arijit Pramanik
Siim Põder
Austin Kyker
Mina Khan
Evgeny Sluzhaev
Marvin Ritter
Avraham Ruderman
Wenlei Zhou
Chirag Nagpal
Kiran Vodrahalli
George Necula
Paul Barham
Ellie Pavlick
Jay Hartford
Izhak Shafran
Long Zhao
Maciej Mikuła
Tom Eccles
Hidetoshi Shimokawa
Kanav Garg
Luke Vilnis
Hanwen Chen
Ilia Shumailov
Kuang-Huei Lee
Abdelrahman Abdelhamed
Meiyan Xie
Vered Cohen
Ester Hlavnova
Dan Malkin
Chawin Sitawarin
James Lottes
Pauline Coquinot
Tianli Yu
Sandeep Kumar
Jingwei Zhang
Aroma Mahendru
Zafarali Ahmed
James Martens
Tao Chen
Aviel Boag
Daiyi Peng
Coline Devin
Arseniy Klimovskiy
Mary Phuong
Danny Vainstein
Jin Xie
Bhuvana Ramabhadran
Nathan Howard
Xinxin Yu
Gitartha Goswami
Jingyu Cui
Sam Shleifer
Mario Pinto
Chih-Kuan Yeh
Ming-Hsuan Yang
Sara Javanmardi
Dan Ethier
Chace Lee
Jordi Orbay
Suyog Kotecha
Carla Bromberg
Pete Shaw
James Thornton
Adi Gerzi Rosenthal
Shane Gu
Matt Thomas
Ian Gemp
Aditya Ayyar
Asahi Ushio
Aarush Selvan
Joel Wee
Chenxi Liu
Maryam Majzoubi
Weiren Yu
Jake Abernethy
Tyler Liechty
Renke Pan
Hoang Nguyen
Qiong
Hu
Sarah Perrin
Abhinav Arora
Emily Pitler
Weiyi Wang
Kaushik Shivakumar
Flavien Prost
Ben Limonchik
Jing Wang
Yi Gao
Timothee Cour
Shyamal Buch
Huan Gui
Maria Ivanova
Philipp Neubeck
Kelvin Chan
Lucy Kim
Huizhong Chen
Naman Goyal
Da-Woon Chung
Lu Liu
Yao Su
Anastasia Petrushkina
Jiajun Shen
Armand Joulin
Yuanzhong Xu
Stein Xudong Lin
Yana Kulizhskaya
Ciprian Chelba
Shobha Vasudevan
Eli Collins
Vasilisa Bashlovkina
Tony Lu
Doug Fritz
Jongbin Park
Yanqi Zhou
Chen Su
Richard Tanburn
Mikhail Sushkov
Mitchelle Rasquinha
Jinning Li
Jennifer Prendki
Yiming Li
Pallavi LV
Shriya Sharma
Hen Fitoussi
Hui Huang
Andrew Dai
Phuong Dao
Mike Burrows
Henry Prior
Danfeng Qin
Golan Pundak
Lars Lowe Sjoesund
Art Khurshudov
Zhenkai Zhu
Albert Webson
Elizabeth Kemp
Tat Tan
Saurabh Agrawal
Susie Sargsyan
Liqun Cheng
Jim Stephan
Tom Kwiatkowski
David Reid
Arunkumar Byravan
Assaf Hurwitz Michaely
Nicolas Heess
Luowei Zhou
Sonam Goenka
Viral Carpenter
Anselm Levskaya
Bo Wang
Reed Roberts
Rémi Leblond
Sharat Chikkerur
Stav Ginzburg
Max Chang
Robert Riachi
Chuqiao
Xu
Zalán Borsos
Michael Pliskin
Julia Pawar
Morgane Lustman
Hannah Kirkwood
Ankit Anand
Aditi Chaudhary
Norbert Kalb
Kieran Milan
Sean Augenstein
Anna Goldie
Laurel Prince
Karthik Raman
Yanhua Sun
Vivian Xia
Aaron Cohen
Zhouyuan Huo
Josh Camp
Seher Ellis
Lukas Zilka
David Vilar Torres
Lisa Patel
Sho Arora
Betty Chan
Jonas Adler
Kareem Ayoub
Jacky Liang
Fayaz Jamil
Jiepu Jiang
Simon Baumgartner
Haitian Sun
Yael Karov
Yaroslav Akulov
Hui Zheng
Irene Cai
Claudio Fantacci
James Rubin
Alex Rav Acha
Mengchao Wang
Nina D'Souza
Rohit Sathyanarayana
Shengyang Dai
Simon Rowe
Andrey Simanovsky
Omer Goldman
Yuheng Kuang
Xiaoyue Pan
Andrew Rosenberg
Tania Rojas-Esponda
Praneet Dutta
Amy Zeng
Irina Jurenka
Greg Farquhar
Yamini Bansal
Shariq Iqbal
Becca Roelofs
Ga-Young Joung
Parker Beak
Changwan Ryu
Ryan Poplin
Yan Wu
Jean-Baptiste Alayrac
Senaka Buthpitiya
Olaf Ronneberger
Caleb Habtegebriel
Wei Li
Paul Cavallaro
Aurora Wei
Guy Bensky
Timo Denk
Harish Ganapathy
Jeff Stanway
Pratik Joshi
Francesco Bertolini
Jessica Lo
Olivia Ma
Zachary Charles
Geta Sampemane
Himanshu Sahni
Xu Chen
Harry Askham
David Gaddy
Peter Young
Jiewen Tan
Matan Eyal
Arthur Bražinskas
Li Zhong
Zhichun Wu
Mark Epstein
Kai Bailey
Andrew Hard
Kamyu Lee
Sasha Goldshtein
Alex Ruiz
Mohammed Badawi
Matthias Lochbrunner
JK Kearns
Ashley Brown
Fabio Pardo
Theophane Weber
Haichuan Yang
Pan-Pan Jiang
Berkin Akin
Zhao Fu
Marcus Wainwright
Chi Zou
Meenu Gaba
Pierre-Antoine Manzagol
Wendy Kan
Yang Song
Karina Zainullina
Rui Lin
Jeongwoo Ko
Salil Deshmukh
Apoorv Jindal
James Svensson
Divya Tyam
Heri Zhao
Christine Kaeser-Chen
Scott Baird
Pooya Moradi
Jamie Hall
Qiuchen Guo
Vincent Tsang
Bowen Liang
Fernando Pereira
Suhas Ganesh
Ivan Korotkov
Jakub Adamek
Sridhar Thiagarajan
Vinh Tran
Charles Chen
Chris Tar
Sanil Jain
Ishita Dasgupta
Taylan Bilal
David Reitter
Kai Zhao
Giulia Vezzani
Yasmin Gehman
Pulkit Mehta
Lauren Beltrone
Xerxes Dotiwalla
Sergio Guadarrama
Zaheer Abbas
Stefani Karp
Petko Georgiev
Chun-Sung Ferng
Marc Brockschmidt
Liqian Peng
Christoph Hirnschall
Vikas Verma
Yingying Bi
Ying Xiao
Avigail Dabush
Kelvin Xu
Phil Wallis
Randall Parker
Qifei Wang
Yang Xu
Ilkin Safarli
Dinesh Tewari
Yin Zhang
Seungyeon Kim
Andrea Gesmundo
Mackenzie Thomas
Sergey Levi
Ahmed Chowdhury
Kanishka Rao
Peter Garst
Sam Conway-Rahman
Helen Ran
Kay McKinney
Zhisheng Xiao
Wenhao Yu
Rohan Agrawal
Axel Stjerngren
Catalin Ionescu
Jingjing Chen
Vivek Sharma
Justin Chiu
Fei Liu
Ken Franko
Clayton Sanford
Xingyu Cai
Paul Michel
Sanjay Ganapathy
Jane Labanowski
Zachary Garrett
Ben Vargas
Sean Sun
Bryan Gale
Thomas Buschmann
Guillaume Desjardins
Nimesh Ghelani
Palak Jain
Mudit Verma
Chulayuth Asawaroengchai
Julian Eisenschlos
Jitendra Harlalka
Hideto Kazawa
Don Metzler
Joshua Howland
Ying Jian
Jake Ades
Viral Shah
Tynan Gangwani
Seungji Lee
Roman Ring
Steven M. Hernandez
Dean Reich
Amer Sinha
Ashutosh Sathe
Joe Kovac
Ashleah Gill
Ajay Kannan
Andrea D'olimpio
Martin Sevenich
Jay Whang
Been Kim
Khe Chai Sim
Jilin Chen
Jiageng Zhang
Shuba Lall
Yossi Matias
Bill Jia
Abe Friesen
Sara Nasso
Ashish Thapliyal
Bryan Perozzi
Ting Yu
Anna Shekhawat
Safeen Huda
Peter Grabowski
Eric Wang
Ashwin Sreevatsa
Hilal Dib
Mehadi Hassen
Parker Schuh
Vedrana Milutinovic
Chris Welty
Michael Quinn
Ali Shah
Bangju Wang
Gabe Barth-Maron
Justin Frye
Natalie Axelsson
Tao Zhu
Yukun Ma
Irene Giannoumis
Hanie Sedghi
Chang Ye
Yi Luan
Kevin Aydin
Bilva Chandra
Vivek Sampathkumar
Ronny Huang
Victor Lavrenko
Ahmed Eleryan
Zhi Hong
Steven Hansen
Sara Mc Carthy
Bidisha Samanta
Domagoj Ćevid
Xin Wang
Fangtao Li
Michael Voznesensky
Matt Hoffman
Andreas Terzis
Vikash Sehwag
Gil Fidel
Luheng He
Mu Cai
Yanzhang He
Alex Feng
Martin Nikoltchev
Samrat Phatale
Jason Chase
Rory Lawton
Ming Zhang
Tom Ouyang
Manuel Tragut
Mehdi Hafezi Manshadi
Arjun Narayanan
Jiaming Shen
Xu Gao
Tolga Bolukbasi
Nick Roy
Xin Li
Daniel Golovin
Liviu Panait
Zhen Qin
Guangxing Han
Thomas Anthony
Sneha Kudugunta
Viorica Patraucean
Aniket Ray
Xinyun Chen
Xiaochen Yang
Tanuj Bhatia
Pranav Talluri
Alex Morris
Andrija Ražnatović
Bethanie Brownfield
James An
Sheng Peng
Patrick Kane
Ce Zheng
Nico Duduta
Joshua Kessinger
James Noraky
Siqi Liu
Keran Rong
Petar Veličković
Keith Rush
Alex Goldin
Fanny Wei
Shiva Mohan Reddy Garlapati
Caroline Pantofaru
Okwan Kwon
Jianmo Ni
Eric Noland
Julia Di Trapani
Françoise Beaufays
Abhijit Guha Roy
Yinlam Chow
Aybuke Turker
Geoffrey Cideron
Lantao Mei
Jon Clark
Qingyun Dou
Matko Bošnjak
Ralph Leith
Yuqing Du
Amir Yazdanbakhsh
Milad Nasr
Chester Kwak
Suraj Satishkumar Sheth
Alex Kaskasoli
Ankesh Anand
Balaji Lakshminarayanan
Sammy Jerome
David Bieber
Chun-Te Chu
Alexandre Senges
Tianxiao Shen
Mukund Sridhar
Ndaba Ndebele
Benjamin Beyret
Shakir Mohamed
Mia Chen
Markus Freitag
Jiaxian Guo
Luyang Liu
Paul Roit
Heng Chen
Shen Yan
Tom Stone
JD Co-Reyes
Jeremy Cole
Salvatore Scellato
Shekoofeh Azizi
Hadi Hashemi
Alicia Jin
Anand Iyer
Marcella Valentine
András György
Arun Ahuja
Daniel Hernandez Diaz
Chen-Yu Lee
Nathan Clement
Weize Kong
Drew Garmon
Ishaan Watts
Kush Bhatia
Khyatti Gupta
Matt Miecnikowski
Hugo Vallet
Ankur Taly
Edward Loper
Saket Joshi
James Atwood
Jo Chick
Mark Collier
Fotis Iliopoulos
Ryan Trostle
Beliz Gunel
Ramiro Leal-Cavazos
Arnar Mar Hrafnkelsson
Michael Guzman
Xiaoen Ju
Andy Forbes
Jesse Emond
Kushal Chauhan
Ben Caine
Li Xiao
Wenjun Zeng
Alexandre Moufarek
Daniel Murphy
Maya Meng
Nitish Gupta
Felix Riedel
Anil Das
Elijah Lawal
Shashi Narayan
Tiberiu Sosea
James Swirhun
Linda Friso
Behnam Neyshabur
Jing Lu
Sertan Girgin
Michael Wunder
Edouard Yvinec
Aroonalok Pyne
Victor Carbune
Shruti Rijhwani
Yang Guo
Tulsee Doshi
Anton Briukhov
Max Bain
Ayal Hitron
Xuanhui Wang
Ashish Gupta
Ke Chen
Cosmo Du
Weiyang Zhang
Dhruv Shah
Arjun Akula
Max Dylla
Ashyana Kachra
Weicheng Kuo
Tingting Zou
Lily Wang
Luyao Xu
Jifan Zhu
Justin Snyder
Sachit Menon
Orhan Firat
Igor Mordatch
Yuan Yuan
Natalia Ponomareva
Rory Blevins
Lawrence Moore
Weijun Wang
Phil Chen
Martin Scholz
Artur Dwornik
Jason Lin
Sicheng Li
Diego Antognini
Te I
Xiaodan Song
Matt Miller
Uday Kalra
Adam Raveret
Oscar Akerlund
Felix Wu
Andrew Nystrom
Namrata Godbole
Tianqi Liu
Hannah DeBalsi
Jewel Zhao
Buhuang Liu
Avi Caciularu
Lauren Lax
Urvashi Khandelwal
Victoria Langston
Eric Bailey
Silvio Lattanzi
Yufei Wang
Neel Kovelamudi
Sneha Mondal
Guru Guruganesh
Nan Hua
Ofir Roval
Paweł Wesołowski
Rishikesh Ingale
Jonathan Halcrow
Tim Sohn
Christof Angermueller
Bahram Raad
Eli Stickgold
Eva Lu
Alec Kosik
Jing Xie
Timothy Lillicrap
Austin Huang
Lydia Lihui Zhang
Dominik Paulus
Clement Farabet
Alex Wertheim
Bing Wang
Rishabh Joshi
Chu-ling Ko
Yonghui Wu
Shubham Agrawal
Lily Lin
XiangHai Sheng
Peter Sung
Tyler Breland-King
Christina Butterfield
Swapnil Gawde
Sumeet Singh
Qiao Zhang
Raj Apte
Shilpa Shetty
Adrian Hutter
Tao Li
Elizabeth Salesky
Federico Lebron
Jonni Kanerva
Michela Paganini
Arthur Nguyen
Rohith Vallu
Jan-Thorsten Peter
Sarmishta Velury
David Kao
Jay Hoover
Anna Bortsova
Colton Bishop
Shoshana Jakobovits
Alessandro Agostini
Alekh Agarwal
Chang Liu
Charles Kwong
Sasan Tavakkol
Ioana Bica
Alex Greve
Anirudh GP
Jake Marcus
Le Hou
Tom Duerig
Rivka Moroshko
Dave Lacey
Andy Davis
Julien Amelot
Guohui Wang
Frank Kim
Theofilos Strinopoulos
Hui Wan
Charline Le Lan
Shankar Krishnan
Haotian Tang
Peter Humphreys
Junwen Bai
Idan Heimlich Shtacher
Diego Machado
Chenxi Pang
Ken Burke
Dangyi Liu
Renga Aravamudhan
Yue Song
Ed Hirst
Abhimanyu Singh
Brendan Jou
Liang Bai
Francesco Piccinno
Chuyuan Kelly Fu
Robin Alazard
Barak Meiri
Daniel Winter
Charlie Chen
Mingda Zhang
Jens Heitkaemper
John Lambert
Jinhyuk Lee
Alexander Frömmgen
Sergey Rogulenko
Pranav Nair
Paul Niemczyk
Anton Bulyenov
Bibo Xu
Hadar Shemtov
Morteza Zadimoghaddam
Serge Toropov
Mateo Wirth
Hanjun Dai
Sreenivas Gollapudi
Daniel Zheng
Alex Kurakin
Chansoo Lee
Kalesha Bullard
Nicolas Serrano
Ivana Balazevic
Yang Li
Johan Schalkwyk
Mark Murphy
Mingyang Zhang
Kevin Sequeira
Romina Datta
Nishant Agrawal
Charles Sutton
Nithya Attaluri
Mencher Chiang
Wael Farhan
Gregory Thornton
Kate Lin
Travis Choma
Hung Nguyen
Kingshuk Dasgupta
Dirk Robinson
Iulia Comşa
Michael Riley
Arjun Pillai
Basil Mustafa
Ben Golan
Amir Zandieh
Jean-Baptiste Lespiau
Billy Porter
David Ross
Sujeevan Rajayogam
Mohit Agarwal
Subhashini Venugopalan
Bobak Shahriari
Qiqi Yan
Hao Xu
Taylor Tobin
Pavel Dubov
Hongzhi Shi
Adrià Recasens
Anton Kovsharov
Sebastian Borgeaud
Lucio Dery
Shanthal Vasanth
Elena Gribovskaya
Linhai Qiu
Mahdis Mahdieh
Wojtek Skut
Elizabeth Nielsen
CJ Zheng
Adams Yu
Carrie Grimes Bostock
Shaleen Gupta
Aaron Archer
Chris Rawles
Elinor Davies
Alexey Svyatkovskiy
Tomy Tsai
Yoni Halpern
Christian Reisswig
Bartek Wydrowski
Bo Chang
Joan Puigcerver
Mor Hazan Taege
Jian Li
Eva Schnider
Xinjian Li
Dragos Dena
Yunhan Xu
Umesh Telang
Tianze Shi
Heiga Zen
Kyle Kastner
Yeongil Ko
Neesha Subramaniam
Aviral Kumar
Pete Blois
Zhuyun Dai
John Wieting
Yifeng Lu
Yoel Zeldes
Tian Xie
Anja Hauth
Alexandru Ţifrea
Yuqi Li
Sam El-Husseini
Dan Abolafia
Howard Zhou
Wen Ding
Sahra Ghalebikesabi
Carlos Guía
Andrii Maksai
Ágoston Weisz
Sercan Arik
Nick Sukhanov
Aga Świetlik
Xuhui Jia
Luo Yu
Weiyue Wang
Mark Brand
Dawn Bloxwich
Sean Kirmani
Zhe Chen
Alec Go
Pablo Sprechmann
Nithish Kannen
Alen Carin
Paramjit Sandhu
Isabel Edkins
Leslie Nooteboom
Jai Gupta
Loren Maggiore
Javad Azizi
Yael Pritch
Pengcheng Yin
Mansi Gupta
Danny Tarlow
Duncan Smith
Desi Ivanov
Mohammad Babaeizadeh
Ankita Goel
Satish Kambala
Grace Chu
Matej Kastelic
Michelle Liu
Hagen Soltau
Austin Stone
Shivani Agrawal
Min Kim
Kedar Soparkar
Srinivas Tadepalli
Oskar Bunyan
Rachel Soh
Arvind Kannan
DY Kim
Blake JianHang Chen
Afief Halumi
Sudeshna Roy
Yulong Wang
Olcan Sercinoglu
Gena Gibson
Sijal Bhatnagar
Motoki Sano
Daniel von Dincklage
Qingchun Ren
Blagoj Mitrevski
Mirek Olšák
Jennifer She
Carl Doersch
Jilei
Wang
Bingyuan Liu
Qijun Tan
Tamar Yakar
Tris Warkentin
Alex Ramirez
Carl Lebsack
Josh Dillon
Rajiv Mathews
Tom Cobley
Zelin Wu
Zhuoyuan Chen
Jon Simon
Swaroop Nath
Tara Sainath
Alexei Bendebury
Ryan Julian
Bharath Mankalale
Daria Ćurko
Paulo Zacchello
Adam R. Brown
Kiranbir Sodhia
Heidi Howard
Sergi Caelles
Abhinav Gupta
Gareth Evans
Anna Bulanova
Lesley Katzen
Roman Goldenberg
Anton Tsitsulin
Joe Stanton
Benoit Schillings
Vitaly Kovalev
Corey Fry
Rushin Shah
Kuo Lin
Shyam Upadhyay
Cheng Li
Soroush Radpour
Marcello Maggioni
Jing Xiong
Lukas Haas
Jenny Brennan
Aishwarya Kamath
Nikolay Savinov
Arsha Nagrani
Trevor Yacovone
Ryan Kappedal
Kostas Andriopoulos
Li Lao
YaGuang Li
Grigory Rozhdestvenskiy
Kazuma Hashimoto
Andrew Audibert
Sophia Austin
Daniel Rodriguez
Anian Ruoss
Garrett Honke
Deep Karkhanis
Xi Xiong
Qing Wei
James Huang
Zhaoqi Leng
Vittal Premachandran
Stan Bileschi
Georgios Evangelopoulos
Thomas Mensink
Jay Pavagadhi
Denis Teplyashin
Paul Chang
Linting Xue
Garrett Tanzer
Sally Goldman
Kaushal Patel
Shixin Li
Jeremy Wiesner
Ivy Zheng
Ian Stewart-Binks
Jie Han
Zhi Li
Liangchen Luo
Karel Lenc
Mario Lučić
Fuzhao Xue
Ryan Mullins
Alexey Guseynov
Chung-Ching Chang
Isaac Galatzer-Levy
Adam Zhang
Garrett Bingham
Grace Hu
Ale Hartman
Yue Ma
Jordan Griffith
Alex Irpan
Carey Radebaugh
Summer Yue
Lijie Fan
Victor Ungureanu
Christina Sorokin
Hannah Teufel
Peiran Li
Rohan Anil
Dimitris Paparas
Todd Wang
Chu-Cheng Lin
Hui Peng
Megan Shum
Goran Petrovic
Demetra Brady
Richard Nguyen
Klaus Macherey
Zhihao Li
Harman Singh
Madhavi Yenugula
Mariko Iinuma
Xinyi Chen
Kavya Kopparapu
Alexey Stern
Shachi Dave
Chandu Thekkath
Florence Perot
Anurag Kumar
Fangda Li
Yang Xiao
Matthew Bilotti
Mohammad Hossein Bateni
Isaac Noble
Lisa Lee
Amelio Vázquez-Reina
Julian Salazar
Xiaomeng Yang
Boyu Wang
Ela Gruzewska
Anand Rao
Sindhu Raghuram
Zheng Xu
Eyal Ben-David
Jieru Mei
Sid Dalmia
Zhaoyi Zhang
Yuchen Liu
Gagan Bansal
Helena Pankov
Steven Schwarcz
Andrea Burns
Christine Chan
Sumit Sanghai
Ricky Liang
Ethan Liang
Antoine He
Amy Stuart
Arun Narayanan
Yukun Zhu
Christian Frank
Bahar Fatemi
Amit Sabne
Oran Lang
Indro Bhattacharya
Shane Settle
Maria Wang
Brendan McMahan
Andrea Tacchetti
Livio Baldini Soares
Majid Hadian
Serkan Cabi
Timothy Chung
Nikita Putikhin
Gang Li
Jeremy Chen
Austin Tarango
Henryk Michalewski
Mehran Kazemi
Hussain Masoom
Hila Sheftel
Rakesh Shivanna
Archita Vadali
Ramona Comanescu
Doug Reid
Joss Moore
Arvind Neelakantan
Michaël Sander
Jonathan Herzig
Aviv Rosenberg
Mostafa Dehghani
JD Choi
Michael Fink
Reid Hayes
Eric Ge
Shitao Weng
Chia-Hua Ho
John Karro
Kalpesh Krishna
Lam Nguyen Thiet
Amy Skerry-Ryan
Daniel Eppens
Marco Andreetto
Navin Sarma
Silvano Bonacina
Burcu Karagol Ayan
Megha Nawhal
Zhihao Shan
Mike Dusenberry
Shantanu Thakoor
Sagar Gubbi
Duc Dung Nguyen
Reut Tsarfaty
Samuel Albanie
Jovana Mitrović
Meet Gandhi
Bo-Juen Chen
Alessandro Epasto
Georgi Stephanov
Ye Jin
Samuel Gehman
Aida Amini
Jack Weber
Feryal Behbahani
Shawn Xu
Miltos Allamanis
Xi Chen
Myle Ott
Claire Sha
Michal Jastrzebski
Hang Qi
David Greene
Xinyi Wu
Abodunrinwa Toki
Daniel Vlasic
Jane Shapiro
Ragha Kotikalapudi
Zhe Shen
Takaaki Saeki
Sirui Xie
Albin Cassirer
Shikhar Bharadwaj
Tatsuya Kiyono
Srinadh Bhojanapalli
Elan Rosenfeld
Sam Ritter
Jieming Mao
João Gabriel Oliveira
Zoltan Egyed
Bernd Bandemer
Emilio Parisotto
Keisuke Kinoshita
Juliette Pluto
Petros Maniatis
Steve Li
Yaohui Guo
Golnaz Ghiasi
Jean Tarbouriech
Srimon Chatterjee
Julie Jin
Katrina
Xu
Jennimaria Palomaki
Séb Arnold
Madhavi Sewak
Federico Piccinini
Mohit Sharma
Ben Albrecht
Sean Purser-haskell
Ashwin Vaswani
Chongyan Chen
Matheus Wisniewski
Qin Cao
John Aslanides
Nguyet Minh Phu
Maximilian Sieb
Lauren Agubuzu
Anne Zheng
Daniel Sohn
Marco Selvi
Anders Andreassen
Krishan Subudhi
Prem Eruvbetine
Oliver Woodman
Tomas Mery
Sebastian Krause
Xiaoqi Ren
Xiao Ma
Jincheng Luo
Dawn Chen
Wei Fan
Henry Griffiths
Christian Schuler
Alice Li
Shujian Zhang
Jean-Michel Sarr
Shixin Luo
Riccardo Patana
Matthew Watson
Dani Naboulsi
Michael Collins
Sailesh Sidhwani
Emiel Hoogeboom
Sharon Silver
Emily Caveness
Xiaokai Zhao
Mikel Rodriguez
Maxine Deines
Libin Bai
Patrick Griffin
Marco Tagliasacchi
Emily Xue
Spandana Raj Babbula
Bo Pang
Nan Ding
Gloria Shen
Elijah Peake
Remi Crocker
Shubha Srinivas Raghvendra
Danny Swisher
Woohyun Han
Richa Singh
Ling Wu
Vladimir Pchelin
Tsendsuren Munkhdalai
Dana Alon
Geoff Bacon
Efren Robles
Jannis Bulian
Melvin Johnson
George Powell
Felipe Tiengo Ferreira
Yaoyiran Li
Frederik Benzing
Mihajlo Velimirović
Hubert Soyer
William Kong
Tony
Nguyên
Zhen Yang
Jeremiah Liu
Joost van Amersfoort
Daniel Gillick
Baochen Sun
Nathalie Rauschmayr
Katie Zhang
Serena Zhan
Tao Zhou
Alexey Frolov
Chengrun Yang
Denis Vnukov
Louis Rouillard
Hongji Li
Amol Mandhane
Nova Fallen
Rajesh Venkataraman
Clara Huiyi Hu
Jennifer Brennan
Jenny Lee
Jerry Chang
Martin Sundermeyer
Zhufeng Pan
Rosemary Ke
Simon Tong
Alex Fabrikant
William Bono
Jindong Gu
Ryan Foley
Yiran Mao
Manolis Delakis
Dhruva Bhaswar
Roy Frostig
Nick Li
Avital Zipori
Cath Hope
Olga Kozlova
Swaroop Mishra
Josip Djolonga
Craig Schiff
Majd Al Merey
Eleftheria Briakou
Peter Morgan
Andy Wan
Avinatan Hassidim
RJ Skerry-Ryan
Kuntal Sengupta
Mary Jasarevic
Praveen Kallakuri
Paige Kunkle
Hannah Brennan
Tom Lieber
Hassan Mansoor
Julian Walker
Bing Zhang
Annie Xie
Goran Žužić
Adaeze Chukwuka
Alex Druinsky
Donghyun Cho
Rui Yao
Ferjad Naeem
Shiraz Butt
Eunyoung Kim
Zhipeng Jia
Mandy Jordan
Adam Lelkes
Mark Kurzeja
Sophie Wang
James Zhao
Andrew Over
Abhishek Chakladar
Marcel Prasetya
Neha Jha
Sriram Ganapathy
Yale Cong
Prakash Shroff
Carl Saroufim
Sobhan Miryoosefi
Mohamed Hammad
Tajwar Nasir
Weijuan Xi
Yang Gao
Young Maeng
Ben Hora
Chin-Yi Cheng
Parisa Haghani
Yoad Lewenberg
Caden Lu
Martin Matysiak
Naina Raisinghani
Huiyu Wang
Lexi Baugher
Rahul Sukthankar
Minh Giang
John Schultz
Noah Fiedel
Minmin Chen
Cheng-Chun Lee
Tapomay Dey
Hao Zheng
Shachi Paul
Celine Smith
Andy Ly
Yicheng Wang
Rishabh Bansal
Bartek Perz
Susanna Ricco
Stasha Blank
Vaishakh Keshava
Deepak Sharma
Marvin Chow
Kunal Lad
Komal Jalan
Simon Osindero
Craig Swanson
Jacob Scott
Anastasija Ilić
Xiaowei Li
Siddhartha Reddy Jonnalagadda
Afzal Shama Soudagar
Yan Xiong
Bat-Orgil Batsaikhan
Daniel Jarrett
Naveen Kumar
Maulik Shah
Matt Lawlor
Austin Waters
Mark Graham
Rhys May
Sabela Ramos
Sandra Lefdal
Zeynep Cankara
Nacho Cano
Brendan O'Donoghue
Jed Borovik
Frederick Liu
Jordan Grimstad
Mahmoud Alnahlawi
Katerina Tsihlas
Tom Hudson
Nikolai Grigorev
Yiling Jia
Terry Huang
Tobenna Peter Igwe
Sergei Lebedev
Xiaodan Tang
Igor Krivokon
Frankie Garcia
Melissa Tan
Eric Jia
Peter Stys
Shikhar Vashishth
Yu Liang
Balaji Venkatraman
Chenjie Gu
Anastasios Kementsietsidis
Chen Zhu
Junehyuk Jung
Yunfei Bai
Mohammad Javad Hosseini
Faruk Ahmed
Aditya Gupta
Xin Yuan
Shereen Ashraf
Shitij Nigam
Gautam Vasudevan
Pranjal Awasthi
Adi Mayrav Gilady
Zelda Mariet
Ramy Eskander
Haiguang Li
Hexiang Hu
Guillermo Garrido
Philippe Schlattner
George Zhang
Rohun Saxena
Petar Dević
Kritika Muralidharan
Ashwin Murthy
Yiqian Zhou
Min Choi
Arissa Wongpanich
Zhengdong Wang
Premal Shah
Yuntao Xu
Yiling Huang
Stephen Spencer
Alice Chen
James Cohan
Junjie Wang
Jonathan Tompson
Junru Wu
Ruba Haroun
Haiqiong Li
Blanca Huergo
Fan Yang
Tongxin Yin
James Wendt
Michael Bendersky
Rahma Chaabouni
Javier Snaider
Johan Ferret
Abhishek Jindal
Tara Thompson
Andrew Xue
Will Bishop
Shubham Milind Phal
Archit Sharma
Yunhsuan Sung
Prabakar Radhakrishnan
Mo Shomrat
Reeve Ingle
Roopali Vij
Justin Gilmer
Mihai Dorin Istin
Sam Sobell
Yang Lu
Emily Nottage
Dorsa Sadigh
Jeremiah Willcock
Tingnan Zhang
Steve Xu
Sasha Brown
Katherine Lee
Gary Wang
Yun Zhu
Yi Tay
Cheolmin Kim
Audrey Gutierrez
Abhanshu Sharma
Yongqin Xian
Sungyong Seo
Claire Cui
Elena Pochernina
Cip Baetu
Krzysztof Jastrzębski
Mimi Ly
Mohamed Elhawaty
Dan Suh
Eren Sezener
Pidong Wang
Nancy Yuen
George Tucker
Jiahao Cai
Zuguang Yang
Cindy Wang
Alex Muzio
Hai Qian
Jae Yoo
Derek Lockhart
Kevin R. McKee
Mandy Guo
Malika Mehrotra
Artur Mendonça
Sanket Vaibhav Mehta
Sherry Ben
Chetan Tekur
Jiaqi Mu
Muye Zhu
Victoria Krakovna
Hongrae Lee
AJ Maschinot
Sébastien Cevey
HyunJeong Choe
Aijun Bai
Hansa Srinivasan
Derek Gasaway
Nick Young
Patrick Siegler
Dan Holtmann-Rice
Vihari Piratla
Kate Baumli
Roey Yogev
Alex Hofer
Hado van Hasselt
Svetlana Grant
Yuri Chervonyi
David Silver
Andrew Hogue
Ayushi Agarwal
Kathie Wang
Preeti Singh
Four Flynn
Josh Lipschultz
Robert David
Lizzetth Bellot
Yao-Yuan Yang
Long Le
Filippo Graziano
Kate Olszewska
Kevin Hui
Akanksha Maurya
Nikos Parotsidis
Weijie Chen
Tayo Oguntebi
Joe Kelley
Anirudh Baddepudi
Johannes Mauerer
Gregory Shaw
Alex Siegman
Lin Yang
Shravya Shetty
Subhrajit Roy
Yunting Song
Wojciech Stokowiec
Ryan Burnell
Omkar Savant
Robert Busa-Fekete
Jin Miao
Samrat Ghosh
Liam MacDermed
Phillip Lippe
Mikhail Dektiarev
Zach Behrman
Fabian Mentzer
Kelvin Nguyen
Meng Wei
Siddharth Verma
Chris Knutsen
Sudeep Dasari
Zhipeng Yan
Petr Mitrichev
Xingyu Wang
Virat Shejwalkar
Jacob Austin
Srinivas Sunkara
Navneet Potti
Yan Virin
Christian Wright
Gaël Liu
Oriana Riva
Etienne Pot
Greg Kochanski
Quoc Le
Gargi Balasubramaniam
Arka Dhar
Yuguo Liao
Adam Bloniarz
Divyansh Shukla
Elizabeth Cole
Jong Lee
Sheng Zhang
Sushant Kafle
Siddharth Vashishtha
Parsa Mahmoudieh
Grace Chen
Raphael Hoffmann
Pranesh Srinivasan
Agustin Dal Lago
Yoav Ben Shalom
Zi Wang
Michael Elabd
Anuj Sharma
Junhyuk Oh
Suraj Kothawade
Maigo Le
Marianne Monteiro
Shentao Yang
Kaiz Alarakyia
Robert Geirhos
Diana Mincu
Håvard Garnes
Hayato Kobayashi
Soroosh Mariooryad
Kacper Krasowiak
Zhixin
Lai
Shibl Mourad
Mingqiu Wang
Fan Bu
Ophir Aharoni
Guanjie Chen
Abhimanyu Goyal
Vadim Zubov
Ankur Bapna
Elahe Dabir
Nisarg Kothari
Kay Lamerigts
Nicola De Cao
Jeremy Shar
Christopher Yew
Nitish Kulkarni
Dre Mahaarachchi
Mandar Joshi
Zhenhai Zhu
Jared Lichtarge
Yichao Zhou
Hannah Muckenhirn
Vittorio Selo
Oriol Vinyals
Peter Chen
Anthony Brohan
Vaibhav Mehta
Sarah Cogan
Ruth Wang
Ty Geri
Wei-Jen Ko
Wei Chen
Fabio Viola
Keshav Shivam
Lisa Wang
Madeleine Clare Elish
Raluca Ada Popa
Sébastien Pereira
Jianqiao Liu
Raphael Koster
Donnie Kim
Gufeng Zhang
Sayna Ebrahimi
Partha Talukdar
Yanyan Zheng
Petra Poklukar
Ales Mikhalap
Dale Johnson
Anitha Vijayakumar
Mark Omernick
Matt Dibb
Ayush Dubey
Qiong Hu
Apurv Suman
Vaibhav Aggarwal
Ilya Kornakov
Fei Xia
Wing Lowe
Alexey Kolganov
Ted Xiao
Vitaly Nikolaev
Steven Hemingray
Bonnie Li
Joana Iljazi
Mikołaj Rybiński
Ballie Sandhu
Peggy Lu
Thang Luong
Rodolphe Jenatton
Vineetha Govindaraj
Hui
Li
Gabriel Dulac-Arnold
Wonpyo Park
Henry Wang
Abhinit Modi
Jean Pouget-Abadie
Kristina Greller
Rahul Gupta
Robert Berry
Prajit Ramachandran
Jinyu Xie
Liam McCafferty
Jianling Wang
Kilol Gupta
Hyeontaek Lim
Blaž Bratanič
Andy Brock
Ilia Akolzin
Jim Sproch
Dan Karliner
Duhyeon Kim
Adrian Goedeckemeyer
Noam Shazeer
Cordelia Schmid
Daniele Calandriello
Parul Bhatia
Krzysztof Choromanski
Ceslee Montgomery
Dheeru Dua
Ana Ramalho
Helen King
Yue Gao
Lynn Nguyen
David Lindner
Divya Pitta
Oleaser Johnson
Khalid Salama
Diego Ardila
Michael Han
Erin Farnese
Seth Odoom
Ziyue Wang
Xiangzhuo Ding
Norman Rink
Ray Smith
Harshal Tushar Lehri
Eden Cohen
Neera Vats
Tong He
Parthasarathy Gopavarapu
Adam Paszke
Miteyan Patel
Wouter Van Gansbeke
Lucia Loher
Luis Castro
Maria Voitovich
Tamara von Glehn
Nelson George
Simon Niklaus
Zach Eaton-Rosen
Nemanja Rakićević
Erik Jue
Sagi Perel
Carrie Zhang
Yuval Bahat
Angéline Pouget
Zhi Xing
Fantine Huot
Ashish Shenoy
Taylor Bos
Vincent Coriou
Bryan Richter
Natasha Noy
Yaqing Wang
Santiago Ontanon
Siyang Qin
Gleb Makarchuk
Demis Hassabis
Zhuowan Li
Mandar Sharma
Kumaran Venkatesan
Iurii Kemaev
Roxanne Daniel
Shiyu Huang
Saloni Shah
Octavio Ponce
Warren
Chen
Manaal Faruqui
Jialin Wu
Slavica Andačić
Szabolcs Payrits
Daniel McDuff
Tom Hume
Yuan Cao
MH Tessler
Qingze Wang
Yinan Wang
Ivor Rendulic
Eirikur Agustsson
Matthew Johnson
Tanya Lando
Andrew Howard
Sri Gayatri Sundara Padmanabhan
Mayank Daswani
Andrea Banino
Michael Kilgore
Jonathan Heek
Ziwei Ji
Alvaro Caceres
Conglong Li
Nora Kassner
Alexey Vlaskin
Zeyu Liu
Alex Grills
Yanhan Hou
Roykrong Sukkerd
Gowoon Cheon
Nishita Shetty
Larisa Markeeva
Piotr Stanczyk
Tejas Iyer
Yuan Gong
Shawn Gao
Keerthana Gopalakrishnan
Tim Blyth
Malcolm Reynolds
Avishkar Bhoopchand
Misha Bilenko
Dero Gharibian
Vicky Zayats
Aleksandra Faust
Abhinav Singh
Min Ma
Hongyang Jiao
Sudheendra Vijayanarasimhan
Lora Aroyo
Vikas Yadav
Sarah Chakera
Ashwin Kakarla
Vilobh Meshram
Karol Gregor
Gabriela Botea
Evan Senter
Dawei Jia
Geza Kovacs
Neha Sharma
Sebastien Baur
Kai Kang
Yifan He
Lin Zhuo
Marija Kostelac
Itay Laish
Songyou Peng
Louis O'Bryan
Daniel Kasenberg
Girish Ramchandra Rao
Edouard Leurent
Biao Zhang
Sage Stevens
Ana Salazar
Ye Zhang
Ivan Lobov
Jake Walker
Allen Porter
Morgan Redshaw
Han Ke
Abhishek Rao
Alex Lee
Hoi Lam
Michael Moffitt
Jaeyoun Kim
Siyuan Qiao
Terry Koo
Robert Dadashi
Xinying Song
Mukund Sundararajan
Peng Xu
Chizu Kawamoto
Yan Zhong
Clara Barbu
Apoorv Reddy
Mauro Verzetti
Leon Li
George Papamakarios
Hanna Klimczak-Plucińska
Mary Cassin
Koray Kavukcuoglu
Rigel Swavely
Alain Vaucher
Jeffrey Zhao
Ross Hemsley
Michael Tschannen
Heming Ge
Gaurav Menghani
Yang Yu
Natalie Ha
Wei He
Xiao Wu
Maggie Song
Rachel Sterneck
Stefan Zinke
Dan A. Calian
Annie Marsden
Alejandro Cruzado Ruiz
Matteo Hessel
Almog Gueta
Benjamin Lee
Brian Farris
Manish Gupta
Yunjie Li
Mohammad Saleh
Vedant Misra
Kefan Xiao
Piermaria Mendolicchio
Gavin Buttimore
Varvara Krayvanova
Nigamaa Nayakanti
Matthew Wiethoff
Yash Pande
Azalia Mirhoseini
Ni Lao
Jasmine Liu
Yiqing Hua
Angie Chen
Yury Malkov
Dmitry Kalashnikov
Shubham Gupta
Kartik Audhkhasi
Yuexiang Zhai
Sudhindra Kopalle
Prateek Jain
Eran Ofek
Clemens Meyer
Khuslen Baatarsukh
Hana Strejček
Jun Qian
James Freedman
Ricardo Figueira
Michal Sokolik
Olivier Bachem
Raymond Lin
Dia Kharrat
Chris Hidey
Pingmei Xu
Dennis Duan
Yin Li
Muge Ersoy
Richard Everett
Kevin Cen
Rebeca Santamaria-Fernandez
Amir Taubenfeld
Ian Mackinnon
Linda Deng
Polina Zablotskaia
Shashank Viswanadha
Shivanker Goel
Damion Yates
Yunxiao Deng
Peter Choy
Mingqing Chen
Abhishek Sinha
Alex Mossin
Yiming Wang
Arthur Szlam
Susan Hao
Paul Kishan Rubenstein
Metin Toksoz-Exley
Miranda Aperghis
Yin Zhong
Junwhan Ahn
Michael Isard
Olivier Lacombe
Florian Luisier
Chrysovalantis Anastasiou
Yogesh Kalley
Utsav Prabhu
Emma Dunleavy
Shaan Bijwadia
Justin Mao-Jones
Kelly Chen
Rama Pasumarthi
Emily Wood
Adil Dostmohamed
Nate Hurley
Jiri Simsa
Alicia Parrish
Mantas Pajarskas
Matt Harvey
Ondrej Skopek
Yony Kochinski
Javier Rey
Verena Rieser
Denny Zhou
Sun Jae Lee
Trilok Acharya
Guowang Li
Joe Jiang
Xiaofan Zhang
Bryant Gipson
Ethan Mahintorabi
Marco Gelmi
Nima Khajehnouri
Angel Yeh
Kayi Lee
Loic Matthey
Leslie Baker
Trang Pham
Han Fu
Alex Pak
Prakhar Gupta
Cristina Vasconcelos
Adam Sadovsky
Brian Walker
Sissie Hsiao
Patrik Zochbauer
Andreea Marzoca
Noam Velan
Junhao Zeng
Gilles Baechler
Danny Driess
Divya Jain
Yanping Huang
Lizzie Tao
John Maggs
Nir Levine
Jon Schneider
Erika Gemzer
Samuel Petit
Shan Han
Zach Fisher
Dustin Zelle
Courtney Biles
Eugene Ie
Asya Fadeeva
Casper Liu
Juliana Vicente Franco
Adrian Collister
Hao Zhang
Renshen Wang
Ruizhe Zhao
Leandro Kieliger
Kurt Shuster
Rui Zhu
Boqing Gong
Lawrence Chan
Ruoxi Sun
Sujoy Basu
Roland Zimmermann
Jamie Hayes
Abhishek Bapna
Jasper Snoek
Weel Yang
Puranjay Datta
Jad Al Abdallah
Kevin Kilgour
Lu Li
SQ Mah
Yennie Jun
Morgane Rivière
Abhijit Karmarkar
Tammo Spalink
Tao Huang
Lucas Gonzalez
Duc-Hieu Tran
Averi Nowak
John Palowitch
Martin Chadwick
Ellie Talius
Harsh Mehta
Thibault Sellam
Philipp Fränken
Massimo Nicosia
Kyle He
Aditya Kini
David Amos
Sugato Basu
Harrison Jobe
Eleni Shaw
Qiantong Xu
Colin Evans
Daisuke Ikeda
Chaochao Yan
Larry Jin
Lun Wang
Sachin Yadav
Ilia Labzovsky
Ramesh Sampath
Ada Ma
Candice Schumann
Aditya Siddhant
Rohin Shah
John Youssef
Rishabh Agarwal
Natalie Dabney
Alessio Tonioni
Moran Ambar
Jing Li
Isabelle Guyon
Benny Li
David Soergel
Boya Fang
Georgi Karadzhov
Cristian Udrescu
Trieu Trinh
Vikas Raunak
Seb Noury
Dee Guo
Sonal Gupta
Mara Finkelstein
Denis Petek
Lihao Liang
Greg Billock
Pei Sun
David Wood
Yiwen Song
Xiaobin Yu
Tatiana Matejovicova
Regev Cohen
Kalyan Andra
David D'Ambrosio
Zhiwei Deng
Vincent Nallatamby
Ebrahim Songhori
Rumen Dangovski
Andrew Lampinen
Pankil Botadra
Adam Hillier
Jiawei Cao
Nagabhushan Baddi
Adhi Kuncoro
Toshihiro Yoshino
Ankit Bhagatwala
Marcáurelio Ranzato
Rylan Schaeffer
Tianlin Liu
Shuai Ye
Obaid Sarvana
John Nham
Chenkai Kuang
Isabel Gao
Jinoo Baek
Shubham Mittal
Ayzaan Wahid
Anita Gergely
Bin Ni
Josh Feldman
Carrie Muir
Pascal Lamblin
Wolfgang Macherey
Ethan Dyer
Logan Kilpatrick
Víctor Campos
Mukul Bhutani
Stanislav Fort
Yanif Ahmad
Aliaksei Severyn
Kleopatra Chatziprimou
Oleksandr Ferludin
Mason Dimarco
Aditya Kusupati
Joe Heyward
Dan Bahir
Kevin Villela
Katie Millican
Dror Marcus
Sanaz Bahargam
Caglar Unlu
Nicholas Roth
Zichuan Wei
Siddharth Gopal
Deepanway Ghoshal
Edward Lee
Sharon Lin
Jennie Lees
Dayeong Lee
Anahita Hosseini
Connie Fan
Seth Neel
Marcus Wu
Yasemin Altun
Honglong Cai
Enrique Piqueras
Josh Woodward
Alessandro Bissacco
Salem Haykal
Mahyar Bordbar
Prasha Sundaram
Sarah Hodkinson
Daniel Toyama
George Polovets
Austin Myers
Anu Sinha
Tomer Levinboim
Kashyap Krishnakumar
Rachita Chhaparia
Tatiana Sholokhova
Nitesh Bharadwaj Gundavarapu
Ganesh Jawahar
Haroon Qureshi
Jieru Hu
Nikola Momchev
Matthew Rahtz
Renjie Wu
Aishwarya P S
Kedar Dhamdhere
Meiqi Guo
Umang Gupta
Ali Eslami
Mariano Schain
Michiel Blokzijl
David Welling
Dave Orr
Levent Bolelli
Nicolas Perez-Nieves
Mikhail Sirotenko
Aman Prasad
Arjun Kar
Borja De Balle Pigem
Tayfun Terzi
Gellért Weisz
Dipankar Ghosh
Aditi Mavalankar
Dhruv Madeka
Kaspar Daugaard
Hartwig Adam
Viraj Shah
Dana Berman
Maggie Tran
Steven Baker
Ewa Andrejczuk
Grishma Chole
Ganna Raboshchuk
Mahdi Mirzazadeh
Thais Kagohara
Shimu Wu
Christian Schallhart
Bernett Orlando
Chen Wang
Alban Rrustemi
Hao Xiong
Hao Liu
Arpi Vezer
Nolan Ramsden
Shuo-yiin Chang
Sidharth Mudgal
Yan Li
Nino Vieillard
Yedid Hoshen
Farooq Ahmad
Ambrose Slone
Amy Hua
Natan Potikha
Mirko Rossini
Jon Stritar
Sushant Prakash
Zifeng Wang
Xuanyi Dong
Alireza Nazari
Efrat Nehoran
Kaan Tekelioglu
Yinxiao Li
Kartikeya Badola
Tom Funkhouser
Yuanzhen Li
Varun Yerram
Ramya Ganeshan
Daniel Formoso
Karol Langner
Tian Shi
Huijian Li
Yumeya Yamamori
Amayika Panda
Alaa Saade
Angelo Scorza Scarpati
Chris Breaux
CJ Carey
Zongwei Zhou
Cho-Jui Hsieh
Sophie Bridgers
Alena Butryna
Nishesh Gupta
Vaibhav Tulsyan
Sanghyun Woo
Evgenii Eltyshev
Will Grathwohl
Chanel Parks
Seth Benjamin
Rina Panigrahy
Shenil Dodhia
Daniel De Freitas
Chris Sauer
Will Song
Ferran Alet
Jackson Tolins
Cosmin Paduraru
Xingyi Zhou
Brian Albert
Zizhao Zhang
Lei Shu
Mudit Bansal
Sarah Nguyen
Amir Globerson
Owen Xiao
James Manyika
Tom Hennigan
Rong Rong
Josip Matak
Anton Bakalov
Ankur Sharma
Danila Sinopalnikov
Andrew Pierson
Stephen Roller
Geoff Brown
Mingcen Gao
Toshiyuki Fukuzawa
Amin Ghafouri
Kenny Vassigh
Iain Barr
Zhicheng Wang
Anna Korsun
Rajesh Jayaram
Lijie Ren
Tim Zaman
Samira Khan
Yana Lunts
Dan Deutsch
Dave Uthus
Nitzan Katz
Masha Samsikova
Amr Khalifa
Nikhil Sethi
Jiao Sun
Luming Tang
Uri Alon
Xianghong Luo
Dian Yu
Abhishek Nayyar
Bryce Petrini
Will Truong
Vincent Hellendoorn
Nikolai Chinaev
Chris Alberti
Wei Wang
Jingcao Hu
Vahab Mirrokni
Ananth Balashankar
Avia Aharon
Aahil Mehta
Ahmet Iscen
Joseph Kready
Lucas Manning
Anhad Mohananey
Yuankai Chen
Anshuman Tripathi
Allen Wu
Igor Petrovski
Dawsen Hwang
Martin Baeuml
Shreyas Chandrakaladharan
Yuan Liu
Rey Coaguila
Maxwell Chen
Sally Ma
Pouya Tafti
Susheel Tatineni
Terry Spitz
Jiayu Ye
Paul Vicol
Mihaela Rosca
Adrià Puigdomènech
Zohar Yahav
Sanjay Ghemawat
Hanzhao Lin
Phoebe Kirk
Zaid Nabulsi
Sergey Brin
Bernd Bohnet
Ken Caluwaerts
Aditya Srikanth Veerubhotla
Dan Zheng
Zihang Dai
Petre Petrov
Yichong Xu
Ramin Mehran
Zhuo Xu
Luisa Zintgraf
Jiho Choi
Spurthi Amba Hombaiah
Romal Thoppilan
Sashank Reddi
Lukasz Lew
Li Li
Kellie Webster
KP Sawhney
Lampros Lamprou
Siamak Shakeri
Mayank Lunayach
Jianmin Chen
Sumit Bagri
Alex Salcianu
Ying Chen
Yani Donchev
Charlotte Magister
Signe Nørly
Vitor Rodrigues
Tomas Izo
Hila Noga
Joe Zou
Thomas Köppe
Wenxuan Zhou
Kenton Lee
Xiangzhu Long
Danielle Eisenbud
Anthony Chen
Connor Schenck
Chi Ming To
Peilin Zhong
Emanuel Taropa
Minh Truong
Omer Levy
Danilo Martins
Zhiyuan Zhang
Christopher Semturs
Kelvin Zhang
Alex Yakubovich
Pol Moreno
Lara McConnaughey
Di Lu
Sam Redmond
Lotte Weerts
Yonatan Bitton
Tiziana Refice
Nicolas Lacasse
Arthur Conmy
Corentin Tallec
Julian Odell
Hannah Forbes-Pollard
Arkadiusz Socala
Jonathan Hoech
Pushmeet Kohli
Alanna Walton
Rui Wang
Mikita Sazanovich
Kexin Zhu
Andrei Kapishnikov
Rich Galt
Matthew Denton
Ben Murdoch
Caitlin Sikora
Kareem Mohamed
Wei Wei
Uri First
Tim McConnell
Luis C. Cobo
James Qin
Thi Avrahami
Daniel Balle
Yu Watanabe
Annie Louis
Adam Kraft
Setareh Ariafar
Yiming Gu
Eugénie Rives
Charles Yoon
Andrei Rusu
James Cobon-Kerr
Chris Hahn
Jiaming Luo
Yuvein
Zhu
Niharika Ahuja
Rodrigo Benenson
Raphaël Lopez Kaufman
Honglin Yu
Lloyd Hightower
Junlin Zhang
Darren Ni
Lisa Anne Hendricks
Gabby Wang
Gal Yona
Lalit Jain
Pablo Barrio
Surya Bhupatiraju
Siva Velusamy
Allan Dafoe
Sebastian Riedel
Tara Thomas
Zhe Yuan
Mathias Bellaiche
Sheena Panthaplackel
Klemen Kloboves
Sarthak Jauhari
Canfer Akbulut
Todor Davchev
Evgeny Gladchenko
David Madras
Aleksandr Chuklin
Tyrone Hill
Quan Yuan
Mukundan Madhavan
Luke Leonhard
Dylan Scandinaro
Qihang Chen
Ning Niu
Arthur Douillard
Bogdan Damoc
Yasumasa Onoe
Fabian Pedregosa
Fred Bertsch
Chas Leichner
Joseph Pagadora
Jonathan Malmaud
Sameera Ponda
Andy Twigg
Oleksii Duzhyi
Jingwei Shen
Miaosen Wang
Roopal Garg
Jing Chen
Utku Evci
Jonathan Lee
Leon Liu
Koji Kojima
Masa Yamaguchi
Arunkumar Rajendran
AJ Piergiovanni
Vinodh Kumar Rajendran
Marco Fornoni
Gabriel Ibagon
Harry Ragan
Sadh MNM Khan
John Blitzer
Andrew Bunner
Guan Sun
Takahiro Kosakai
Scott Lundberg
Ndidi Elue
Kelvin Guu
SK Park
Jane Park
Arunachalam Narayanaswamy
Chengda Wu
Jayaram Mudigonda
Trevor Cohn
Hairong Mu
Ravi Kumar
Laura Graesser
Yichi Zhang
Richard Killam
Vincent Zhuang
Mai Giménez
Wael Al Jishi
Ruy Ley-Wild
Alex Zhai
Kazuki Osawa
Diego Cedillo
Jialu Liu
Mayank Upadhyay
Marcin Sieniek
Roshan Sharma
Tom Paine
Anelia Angelova
Sravanti Addepalli
Carolina Parada
Kingshuk Majumder
Avery Lamp
Sanjiv Kumar
Xiang Deng
Artiom Myaskovsky
Tea Sabolić
Jeffrey Dudek
Sarah York
Félix de Chaumont Quitry
Jiazhong Nie
Dee Cattle
Alok Gunjan
Bilal Piot
Waleed Khawaja
Seojin Bang
Simon Wang
Siavash Khodadadeh
Raghavender R
Praynaa Rawlani
Richard Powell
Kevin Lee
Johannes Griesser
GS Oh
Cesar Magalhaes
Yujia Li
Simon Tokumine
Hadas Natalie Vogel
Dennis Hsu
Arturo BC
Disha Jindal
Matan Cohen
Zi Yang
Junwei Yuan
Dario de Cesare
Tony Bruguier
Jun Xu
Monica Roy
Alon Jacovi
Dan Belov
Rahul Arya
Phoenix Meadowlark
Shlomi Cohen-Ganor
Wenting Ye
Patrick Morris-Suzuki
Praseem Banzal
Gan Song
Pranavaraj Ponnuramu
Fred Zhang
George Scrivener
Salah Zaiem
Alif Raditya Rochman
Kehang Han
Badih Ghazi
Kate Lee
Shahar Drath
Daniel Suo
Antonious Girgis
Pradeep Shenoy
Duy Nguyen
Douglas Eck
Somit Gupta
Le Yan
Joao Carreira
Anmol Gulati
Ruoxin Sang
Daniil Mirylenka
Emma Cooney
Edward Chou
Mingyang Ling
Cindy Fan
Ben Coleman
Guilherme Tubone
Ravin Kumar
Jason Baldridge
Felix Hernandez-Campos
Angeliki Lazaridou
James Besley
Itay Yona
Neslihan Bulut
Quentin Wellens
AJ Pierigiovanni
Jasmine George
Richard Green
Pu Han
Connie Tao
Geoff Clark
Chong You
Abbas Abdolmaleki
Justin Fu
Tongzhou Chen
Ashwin Chaugule
Angad Chandorkar
Altaf Rahman
Will Thompson
Penporn Koanantakool
Mike Bernico
Jie Ren
Andrey Vlasov
Sergei Vassilvitskii
Maciej Kula
Yizhong Liang
Dahun Kim
Yangsibo Huang
Chengxi Ye
Dmitry Lepikhin
Wesley Helmholz


When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework

Abstract: We investigate the challenge of applying Large Language Models (LLMs) to long texts. We propose a theoretical framework that distinguishes the failure modes of long context tasks into three categories: cross-chunk dependence (task noise), confusion that grows with context size (model noise), and the imperfect integration of partial results (aggregator noise). Under this view, we analyze when it is effective to use multi-agent chunking, i.e., dividing a lengthy sequence into smaller chunks and aggregating the processed results of each chunk. Our experiments on tasks such as retrieval, question answering, and summarization confirm both the theoretical analysis and the conditions that favor multi-agent chunking. By exploring the accelerated decay of model fidelity with input length, we also explain why, for large inputs, a weaker model configured with chunk-based processing can surpass a more advanced model like GPT4o applied in a single shot. Overall, we present a principled understanding framework and our results highlight a direct pathway to handling long contexts in LLMs with carefully managed chunking and aggregator strategies.

Zhen Xu
Shang Zhu
Jue Wang
Junlin Wang
Ben Athiwaratkun
Chi Wang
James Zou
Ce Zhang


LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs

Abstract: Large Language Diffusion Models, or diffusion LLMs, have emerged as a significant focus in NLP research, with substantial effort directed toward understanding their scalability and downstream task performance. However, their long-context capabilities remain unexplored, lacking systematic analysis or methods for context extension. In this work, we present the first systematic investigation comparing the long-context performance of diffusion LLMs and traditional auto-regressive LLMs. We first identify a unique characteristic of diffusion LLMs, unlike auto-regressive LLMs, they maintain remarkably stable perplexity during direct context extrapolation. Moreover, where auto-regressive models fail outright during the Needle-In-A-Haystack task with context exceeding their pretrained length, we discover diffusion LLMs exhibit a distinct local perception phenomenon, enabling successful retrieval from recent context segments. We explain both phenomena through the lens of Rotary Position Embedding (RoPE) scaling theory. Building on these observations, we propose LongLLaDA, a training-free method that integrates LLaDA with the NTK-based RoPE extrapolation. Our results validate that established extrapolation scaling laws remain effective for extending the context windows of diffusion LLMs. Furthermore, we identify long-context tasks where diffusion LLMs outperform auto-regressive LLMs and others where they fall short. Consequently, this study establishes the first length extrapolation method for diffusion LLMs while providing essential theoretical insights and empirical benchmarks critical for advancing future research on long-context diffusion LLMs. The code is available at https://github.com/OpenMOSS/LongLLaDA.

Xiaoran Liu
Yuerong Song
Zhigeng Liu
Zengfeng Huang
Qipeng Guo
Ziwei He
Xipeng Qiu