new

Get trending papers in your email inbox!

Subscribe

Daily Papers

byAK and the research community

Aug 24

Moir: Let the Model Direct Its Own Story for Robust Cross-Domain Knowledge Editing

While language models remain frozen at their training state, the world evolves continuously. Knowledge editing has emerged as a key alternative to full retraining, but its deployment is bottlenecked by the erosion of core capabilities: mathematical and programmatic reasoning collapse while encyclopedic recall remains intact. We trace this asymmetric degradation to a distributional mismatch. Covariance-based editors preserve only the subspaces spanned by their reference corpus, but fail to capture the operative distribution shaped by post-training such as SFT and DPO. Static external corpora, including Wikipedia and even the original pretraining mixture, cannot recover this shifted manifold. We propose Moir, which estimates the preservation covariance C directly from the model itself by sampling from its own decoding distribution. Seeding generation with a single random vocabulary token bypasses the instruction-following templates that otherwise dominate sampled outputs, exposing the broader subspaces the model has internalized. Moir requires no external data and serves as a drop-in component for any covariance-based editor, a practical advantage given that the pre- and post-training corpora of most modern LLMs are not publicly accessible. Across OLMo-2, Llama-3.1, and Qwen-3 (7-8B), under both MEMIT and AlphaEdit and in batch and sequential regimes, Moir consistently extends preservation in the most vulnerable domains, most strikingly on Qwen3-8B after 20,000 AlphaEdit batch edits, it retains 79.9% GSM8K accuracy compared to 10.9% with the Wikipedia baseline. These results suggest that aligning the preservation distribution with the model's operative distribution is a key factor in non-destructive editing, and that the model itself may be the most accessible source of that distribution for deployed systems.

  • 4 authors
·
May 10

Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts

Time series foundation models have demonstrated impressive performance as zero-shot forecasters. However, achieving effectively unified training on time series remains an open challenge. Existing approaches introduce some level of model specialization to account for the highly heterogeneous nature of time series data. For instance, Moirai pursues unified training by employing multiple input/output projection layers, each tailored to handle time series at a specific frequency. Similarly, TimesFM maintains a frequency embedding dictionary for this purpose. We identify two major drawbacks to this human-imposed frequency-level model specialization: (1) Frequency is not a reliable indicator of the underlying patterns in time series. For example, time series with different frequencies can display similar patterns, while those with the same frequency may exhibit varied patterns. (2) Non-stationarity is an inherent property of real-world time series, leading to varied distributions even within a short context window of a single time series. Frequency-level specialization is too coarse-grained to capture this level of diversity. To address these limitations, this paper introduces Moirai-MoE, using a single input/output projection layer while delegating the modeling of diverse time series patterns to the sparse mixture of experts (MoE) within Transformers. With these designs, Moirai-MoE reduces reliance on human-defined heuristics and enables automatic token-level specialization. Extensive experiments on 39 datasets demonstrate the superiority of Moirai-MoE over existing foundation models in both in-distribution and zero-shot scenarios. Furthermore, this study conducts comprehensive model analyses to explore the inner workings of time series MoE foundation models and provides valuable insights for future research.

  • 10 authors
·
Oct 14, 2024

MOIRCS Deep Survey. X. Evolution of Quiescent Galaxies as a Function of Stellar Mass at 0.5<z<2.5

We study the evolution of quiescent galaxies at 0.5<z<2.5 as a function of stellar mass, using very deep NIR imaging data taken with the Multi-Object Infrared Camera and Spectrograph on the Subaru Telescope in the GOODS-North region. The deep NIR data allow us to construct a stellar mass-limited sample of quiescent galaxies down to ~10^{10} Msun even at z~2 for the first time. We selected quiescent galaxies with age/tau>6 by performing SED fitting of the multi broad-band photometry from the U to Spitzer 5.8um bands with the population synthesis model of Bruzual & Charlot (2003) where exponentially decaying star formation histories are assumed. The number density of quiescent galaxies increases by a factor of ~3 from 1.0<z<1.5 to 0.5<z<1.0, and by a factor of ~10 from 1.5<z<2.5 to 0.5<z<1.0, while that of star-forming galaxies with age/tau<4 increases only by factors of ~2 and ~3 in the same redshift ranges. At 0.5<z<2.5, the low-mass slope of the stellar mass function of quiescent galaxies is alpha ~ 0 -- 0.6, which is significantly flatter than those of star-forming galaxies (alpha ~ -1.3 -- -1.5). As a result, the fraction of quiescent galaxies in the overall galaxy population increases with stellar mass in the redshift range. The fraction of quiescent galaxies at 10^{11}-10^{11.5} Msun increases from ~20-30% at z~2 to ~40-60% at z~0.75, while that at 10^{10}-10^{10.5} Msun increases from <~ 5% to ~15% in the same redshift range. These results could suggest that the quenching of star formation had been more effective in more massive galaxies at 1<~z<~2. Such a mass-dependent quenching could explain the rapid increase of the number density of ~M* galaxies relative to lower-mass galaxies at z >~ 1-1.5.

  • 7 authors
·
Jan 19, 2011

Improving Complex Moiré Removal with Generative Supervision

The availability of high-quality paired data is essential for training learning-based image demoiréing models. However, it remains challenging for existing datasets to encompass the complex moiré patterns captured in uncontrolled real-world scenarios. Such degradations typically manifest as large-scale, multicolored moiré patterns. Moreover, these patterns frequently occur in images for which clean counterparts are difficult to obtain, such as photographs acquired from public displays or existing online resources. In this work, we propose a novel data engine designed to improve the removal of complex moiré patterns by generating training supervision. Specifically, we initially collect real-world images containing complex moiré patterns and localize the corresponding screen regions. Multiple image-conditioned generative foundation models are subsequently deployed to produce candidate references. To establish reliable supervision, these candidates are subjected to patch-level quality control to filter and select the optimal results. Based on this systematic paradigm, we construct the WildMoiré dataset, which contains 6.8K moiré-GT training pairs. For evaluation, we additionally build an independent test set comprising sim250 pairs with captured clean ground truth. Extensive experiments on ESDNet, SDXL, and Qwen-Image-Edit demonstrate that the proposed generative supervision consistently improves the performance of complex moiré removal.

  • 6 authors
·
Aug 17

The X-ray Integral Field Unit at the end of the Athena reformulation phase

The Athena mission entered a redefinition phase in July 2022, driven by the imperative to reduce the mission cost at completion for the European Space Agency below an acceptable target, while maintaining the flagship nature of its science return. This notably called for a complete redesign of the X-ray Integral Field Unit (X-IFU) cryogenic architecture towards a simpler active cooling chain. Passive cooling via successive radiative panels at spacecraft level is now used to provide a 50 K thermal environment to an X-IFU owned cryostat. 4.5 K cooling is achieved via a single remote active cryocooler unit, while a multi-stage Adiabatic Demagnetization Refrigerator ensures heat lift down to the 50 mK required by the detectors. Amidst these changes, the core concept of the readout chain remains robust, employing Transition Edge Sensor microcalorimeters and a SQUID-based Time-Division Multiplexing scheme. Noteworthy is the introduction of a slower pixel. This enables an increase in the multiplexing factor (from 34 to 48) without compromising the instrument energy resolution, hence keeping significant system margins to the new 4 eV resolution requirement. This allows reducing the number of channels by more than a factor two, and thus the resource demands on the system, while keeping a 4' field of view (compared to 5' before). In this article, we will give an overview of this new architecture, before detailing its anticipated performances. Finally, we will present the new X-IFU schedule, with its short term focus on demonstration activities towards a mission adoption in early 2027.

  • 282 authors
·
Feb 15, 2025