FD-Vision Mamba for Endoscopic Exposure Correction

Abstract: In endoscopic imaging, the recorded images are prone to exposure abnormalities, so maintaining high-quality images is important to assist healthcare professionals in performing decision-making. To overcome this issue, We design a frequency-domain based network, called FD-Vision Mamba (FDVM-Net), which achieves high-quality image exposure correction by reconstructing the frequency domain of endoscopic images. Specifically, inspired by the State Space Sequence Models (SSMs), we develop a C-SSM block that integrates the local feature extraction ability of the convolutional layer with the ability of the SSM to capture long-range dependencies. A two-path network is built using C-SSM as the basic function cell, and these two paths deal with the phase and amplitude information of the image, respectively. Finally, a degraded endoscopic image is reconstructed by FDVM-Net to obtain a high-quality clear image. Extensive experimental results demonstrate that our method achieves state-of-the-art results in terms of speed and accuracy, and it is noteworthy that our method can enhance endoscopic images of arbitrary resolution. The URL of the code is \url{https://github.com/zzr-idam/FDVM-Net}.


Mamba-ND: Selective State Space Modeling for Multi-Dimensional Data

Abstract: In recent years, Transformers have become the de-facto architecture for sequence modeling on text and a variety of multi-dimensional data, such as images and video. However, the use of self-attention layers in a Transformer incurs prohibitive compute and memory complexity that scales quadratically w.r.t. the sequence length. A recent architecture, Mamba, based on state space models has been shown to achieve comparable performance for modeling text sequences, while scaling linearly with the sequence length. In this work, we present Mamba-ND, a generalized design extending the Mamba architecture to arbitrary multi-dimensional data. Our design alternatively unravels the input data across different dimensions following row-major orderings. We provide a systematic comparison of Mamba-ND with several other alternatives, based on prior multi-dimensional extensions such as Bi-directional LSTMs and S4ND. Empirically, we show that Mamba-ND demonstrates performance competitive with the state-of-the-art on a variety of multi-dimensional benchmarks, including ImageNet-1K classification, HMDB-51 action recognition, and ERA5 weather forecasting.


Mamba-UNet: UNet-Like Pure Visual Mamba for Medical Image Segmentation

Abstract: In recent advancements in medical image analysis, Convolutional Neural Networks (CNN) and Vision Transformers (ViT) have set significant benchmarks. While the former excels in capturing local features through its convolution operations, the latter achieves remarkable global context understanding by leveraging self-attention mechanisms. However, both architectures exhibit limitations in efficiently modeling long-range dependencies within medical images, which is a critical aspect for precise segmentation. Inspired by the Mamba architecture, known for its proficiency in handling long sequences and global contextual information with enhanced computational efficiency as a State Space Model (SSM), we propose Mamba-UNet, a novel architecture that synergizes the U-Net in medical image segmentation with Mamba's capability. Mamba-UNet adopts a pure Visual Mamba (VMamba)-based encoder-decoder structure, infused with skip connections to preserve spatial information across different scales of the network. This design facilitates a comprehensive feature learning process, capturing intricate details and broader semantic contexts within medical images. We introduce a novel integration mechanism within the VMamba blocks to ensure seamless connectivity and information flow between the encoder and decoder paths, enhancing the segmentation performance. We conducted experiments on publicly available ACDC MRI Cardiac segmentation dataset, and Synapse CT Abdomen segmentation dataset. The results show that Mamba-UNet outperforms several types of UNet in medical image segmentation under the same hyper-parameter setting. The source code and baseline implementations are available.

Ziyang Wang
Jian-Qing Zheng
Yichi Zhang
Ge Cui
Lei Li


Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks

Abstract: State-space models (SSMs), such as Mamba (Gu & Dao, 2023), have been proposed as alternatives to Transformer networks in language modeling, by incorporating gating, convolutions, and input-dependent token selection to mitigate the quadratic cost of multi-head attention. Although SSMs exhibit competitive performance, their in-context learning (ICL) capabilities, a remarkable emergent property of modern language models that enables task execution without parameter optimization, remain underexplored compared to Transformers. In this study, we evaluate the ICL performance of SSMs, focusing on Mamba, against Transformer models across various tasks. Our results show that SSMs perform comparably to Transformers in standard regression ICL tasks, while outperforming them in tasks like sparse parity learning. However, SSMs fall short in tasks involving non-standard retrieval functionality. To address these limitations, we introduce a hybrid model, MambaFormer, that combines Mamba with attention blocks, surpassing individual models in tasks where they struggle independently. Our findings suggest that hybrid architectures offer promising avenues for enhancing ICL in language models.

Jongho Park
Jaeseung Park
Zheyang Xiong
Nayoung Lee
Jaewoong Cho
Samet Oymak
Kangwook Lee
Dimitris Papailiopoulos

International Conference on Machine Learning


Is Mamba Capable of In-Context Learning?

Abstract: State of the art foundation models such as GPT-4 perform surprisingly well at in-context learning (ICL), a variant of meta-learning concerning the learned ability to solve tasks during a neural network forward pass, exploiting contextual information provided as input to the model. This useful ability emerges as a side product of the foundation model's massive pretraining. While transformer models are currently the state of the art in ICL, this work provides empirical evidence that Mamba, a newly proposed state space model which scales better than transformers w.r.t. the input sequence length, has similar ICL capabilities. We evaluated Mamba on tasks involving simple function approximation as well as more complex natural language processing problems. Our results demonstrate that, across both categories of tasks, Mamba closely matches the performance of transformer models for ICL. Further analysis reveals that, like transformers, Mamba appears to solve ICL problems by incrementally optimizing its internal representations. Overall, our work suggests that Mamba can be an efficient alternative to transformers for ICL tasks involving long input sequences. This is an exciting finding in meta-learning and may enable generalizations of in-context learned AutoML algorithms (like TabPFN or Optformer) to long input sequences.

Riccardo Grazzi
Julien Siems
Simon Schrodi
Thomas Brox
Frank Hutter


VM-UNet: Vision Mamba UNet for Medical Image Segmentation

Abstract: In the realm of medical image segmentation, both CNN-based and Transformer-based models have been extensively explored. However, CNNs exhibit limitations in long-range modeling capabilities, whereas Transformers are hampered by their quadratic computational complexity. Recently, State Space Models (SSMs), exemplified by Mamba, have emerged as a promising approach. They not only excel in modeling long-range interactions but also maintain a linear computational complexity. In this paper, leveraging state space models, we propose a U-shape architecture model for medical image segmentation, named Vision Mamba UNet (VM-UNet). Specifically, the Visual State Space (VSS) block is introduced as the foundation block to capture extensive contextual information, and an asymmetrical encoder-decoder structure is constructed with fewer convolution layers to save calculation cost. We conduct comprehensive experiments on the ISIC17, ISIC18, and Synapse datasets, and the results indicate that VM-UNet performs competitively in medical image segmentation tasks. To our best knowledge, this is the first medical image segmentation model constructed based on the pure SSM-based model. We aim to establish a baseline and provide valuable insights for the future development of more efficient and effective SSM-based segmentation systems. Our code is available at https://github.com/JCruan519/VM-UNet.

Jiacheng Ruan
Jincheng Li
Suncheng Xiang

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)


Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spaces

Abstract: Attention mechanisms have been widely used to capture long-range dependencies among nodes in Graph Transformers. Bottlenecked by the quadratic computational cost, attention mechanisms fail to scale in large graphs. Recent improvements in computational efficiency are mainly achieved by attention sparsification with random or heuristic-based graph subsampling, which falls short in data-dependent context reasoning. State space models (SSMs), such as Mamba, have gained prominence for their effectiveness and efficiency in modeling long-range dependencies in sequential data. However, adapting SSMs to non-sequential graph data presents a notable challenge. In this work, we introduce Graph-Mamba, the first attempt to enhance long-range context modeling in graph networks by integrating a Mamba block with the input-dependent node selection mechanism. Specifically, we formulate graph-centric node prioritization and permutation strategies to enhance context-aware reasoning, leading to a substantial improvement in predictive performance. Extensive experiments on ten benchmark datasets demonstrate that Graph-Mamba outperforms state-of-the-art methods in long-range graph prediction tasks, with a fraction of the computational cost in both FLOPs and GPU memory consumption. The code and models are publicly available at https://github.com/bowang-lab/Graph-Mamba.


BlackMamba: Mixture of Experts for State-Space Models

Abstract: State-space models (SSMs) have recently demonstrated competitive performance to transformers at large-scale language modeling benchmarks while achieving linear time and memory complexity as a function of sequence length. Mamba, a recently released SSM model, shows impressive performance in both language modeling and long sequence processing tasks. Simultaneously, mixture-of-expert (MoE) models have shown remarkable performance while significantly reducing the compute and latency costs of inference at the expense of a larger memory footprint. In this paper, we present BlackMamba, a novel architecture that combines the Mamba SSM with MoE to obtain the benefits of both. We demonstrate that BlackMamba performs competitively against both Mamba and transformer baselines, and outperforms in inference and training FLOPs. We fully train and open-source 340M/1.5B and 630M/2.8B BlackMamba models on 300B tokens of a custom dataset. We show that BlackMamba inherits and combines both of the benefits of SSM and MoE architectures, combining linear-complexity generation from SSM with cheap and fast inference from MoE. We release all weights, checkpoints, and inference code open-source. Inference code at: https://github.com/Zyphra/BlackMamba

Quentin Anthony
Yury Tokpanov
Paolo Glorioso
Beren Millidge


MambaByte: Token-free Selective State Space Model

Abstract: Token-free language models learn directly from raw bytes and remove the inductive bias of subword tokenization. Operating on bytes, however, results in significantly longer sequences. In this setting, standard autoregressive Transformers scale poorly as the effective memory required grows with sequence length. The recent development of the Mamba state space model (SSM) offers an appealing alternative approach with a fixed-sized memory state and efficient decoding. We propose MambaByte, a token-free adaptation of the Mamba SSM trained autoregressively on byte sequences. In terms of modeling, we show MambaByte to be competitive with, and even to outperform, state-of-the-art subword Transformers on language modeling tasks while maintaining the benefits of token-free language models, such as robustness to noise. In terms of efficiency, we develop an adaptation of speculative decoding with tokenized drafting and byte-level verification. This results in a $2.6\times$ inference speedup to the standard MambaByte implementation, showing similar decoding efficiency as the subword Mamba. These findings establish the viability of SSMs in enabling token-free language modeling.

Junxiong Wang
Tushaar Gangavarapu
Jing Nathan Yan
Alexander M. Rush


SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentation

Abstract: The Transformer architecture has shown a remarkable ability in modeling global relationships. However, it poses a significant computational challenge when processing high-dimensional medical images. This hinders its development and widespread adoption in this task. Mamba, as a State Space Model (SSM), recently emerged as a notable manner for long-range dependencies in sequential modeling, excelling in natural language processing filed with its remarkable memory efficiency and computational speed. Inspired by its success, we introduce SegMamba, a novel 3D medical image \textbf{Seg}mentation \textbf{Mamba} model, designed to effectively capture long-range dependencies within whole volume features at every scale. Our SegMamba, in contrast to Transformer-based methods, excels in whole volume feature modeling from a state space model standpoint, maintaining superior processing speed, even with volume features at a resolution of {$64\times 64\times 64$}. Comprehensive experiments on the BraTS2023 dataset demonstrate the effectiveness and efficiency of our SegMamba. The code for SegMamba is available at: https://github.com/ge-xing/SegMamba

Zhaohu Xing
Tian Ye
Yijun Yang
Guang Liu
Lei Zhu

International Conference on Medical Image Computing and Computer-Assisted Intervention


Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

Abstract: Recently the state space models (SSMs) with efficient hardware-aware designs, i.e., the Mamba deep learning model, have shown great potential for long sequence modeling. Meanwhile building efficient and generic vision backbones purely upon SSMs is an appealing direction. However, representing visual data is challenging for SSMs due to the position-sensitivity of visual data and the requirement of global context for visual understanding. In this paper, we show that the reliance on self-attention for visual representation learning is not necessary and propose a new generic vision backbone with bidirectional Mamba blocks (Vim), which marks the image sequences with position embeddings and compresses the visual representation with bidirectional state space models. On ImageNet classification, COCO object detection, and ADE20k semantic segmentation tasks, Vim achieves higher performance compared to well-established vision transformers like DeiT, while also demonstrating significantly improved computation & memory efficiency. For example, Vim is 2.8$\times$ faster than DeiT and saves 86.8% GPU memory when performing batch inference to extract features on images with a resolution of 1248$\times$1248. The results demonstrate that Vim is capable of overcoming the computation & memory constraints on performing Transformer-style understanding for high-resolution images and it has great potential to be the next-generation backbone for vision foundation models. Code is available at https://github.com/hustvl/Vim.

Lianghui Zhu
Bencheng Liao
Qian Zhang
Xinlong Wang
Wenyu Liu
Xinggang Wang


Cross Domain Early Crop Mapping using CropSTGAN

Abstract: Driven by abundant satellite imagery, machine learning-based approaches have recently been promoted to generate high-resolution crop cultivation maps to support many agricultural applications. One of the major challenges faced by these approaches is the limited availability of ground truth labels. In the absence of ground truth, existing work usually adopts the "direct transfer strategy" that trains a classifier using historical labels collected from other regions and then applies the trained model to the target region. Unfortunately, the spectral features of crops exhibit inter-region and inter-annual variability due to changes in soil composition, climate conditions, and crop progress, the resultant models perform poorly on new and unseen regions or years. Despite recent efforts, such as the application of the deep adaptation neural network (DANN) model structure in the deep adaptation crop classification network (DACCN), to tackle the above cross-domain challenges, their effectiveness diminishes significantly when there is a large dissimilarity between the source and target regions. This paper introduces the Crop Mapping Spectral-temporal Generative Adversarial Neural Network (CropSTGAN), a novel solution for cross-domain challenges, that doesn't require target domain labels. CropSTGAN learns to transform the target domain's spectral features to those of the source domain, effectively bridging large dissimilarities. Additionally, it employs an identity loss to maintain the intrinsic local structure of the data. Comprehensive experiments across various regions and years demonstrate the benefits and effectiveness of the proposed approach. In experiments, CropSTGAN is benchmarked against various state-of-the-art (SOTA) methods. Notably, CropSTGAN significantly outperforms these methods in scenarios with large data distribution dissimilarities between the target and source domains.


MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

Abstract: State Space Models (SSMs) have become serious contenders in the field of sequential modeling, challenging the dominance of Transformers. At the same time, Mixture of Experts (MoE) has significantly improved Transformer-based Large Language Models, including recent state-of-the-art open models. We propose that to unlock the potential of SSMs for scaling, they should be combined with MoE. We showcase this on Mamba, a recent SSM-based model that achieves remarkable performance. Our model, MoE-Mamba, outperforms both Mamba and baseline Transformer-MoE. In particular, MoE-Mamba reaches the same performance as Mamba in $2.35\times$ fewer training steps while preserving the inference performance gains of Mamba against Transformer.

Maciej Pióro
Kamil Ciebiera
Krystian Król
Jan Ludziejewski
Michał Krutul
Jakub Krajewski
Szymon Antoniak
Piotr Miłoś
Marek Cygan
Sebastian Jaszczur


Self-supervised Machine Learning Based Approach to Orbit Modelling Applied to Space Traffic Management

Abstract: This paper presents a novel methodology for improving the performance of machine learning based space traffic management tasks through the use of a pre-trained orbit model. Taking inspiration from BERT-like self-supervised language models in the field of natural language processing, we introduce ORBERT, and demonstrate the ability of such a model to leverage large quantities of readily available orbit data to learn meaningful representations that can be used to aid in downstream tasks. As a proof of concept of this approach we consider the task of all vs. all conjunction screening, phrased here as a machine learning time series classification task. We show that leveraging unlabelled orbit data leads to improved performance, and that the proposed approach can be particularly beneficial for tasks where the availability of labelled data is limited.

Emma Stevenson
Victor Rodriguez-Fernandez
Hodei Urrutxua
Vincent Morand
David Camacho


Mamba: Linear-Time Sequence Modeling with Selective State Spaces

Abstract: Foundation models, now powering most of the exciting applications in deep learning, are almost universally based on the Transformer architecture and its core attention module. Many subquadratic-time architectures such as linear attention, gated convolution and recurrent models, and structured state space models (SSMs) have been developed to address Transformers' computational inefficiency on long sequences, but they have not performed as well as attention on important modalities such as language. We identify that a key weakness of such models is their inability to perform content-based reasoning, and make several improvements. First, simply letting the SSM parameters be functions of the input addresses their weakness with discrete modalities, allowing the model to selectively propagate or forget information along the sequence length dimension depending on the current token. Second, even though this change prevents the use of efficient convolutions, we design a hardware-aware parallel algorithm in recurrent mode. We integrate these selective SSMs into a simplified end-to-end neural network architecture without attention or even MLP blocks (Mamba). Mamba enjoys fast inference (5$\times$ higher throughput than Transformers) and linear scaling in sequence length, and its performance improves on real data up to million-length sequences. As a general sequence model backbone, Mamba achieves state-of-the-art performance across several modalities such as language, audio, and genomics. On language modeling, our Mamba-3B model outperforms Transformers of the same size and matches Transformers twice its size, both in pretraining and downstream evaluation.


GateLoop: Fully Data-Controlled Linear Recurrence for Sequence Modeling

Abstract: Linear Recurrence has proven to be a powerful tool for modeling long sequences efficiently. In this work, we show that existing models fail to take full advantage of its potential. Motivated by this finding, we develop GateLoop, a foundational sequence model that generalizes linear recurrent models such as S4, S5, LRU and RetNet, by employing data-controlled state transitions. Utilizing this theoretical advance, GateLoop empirically outperforms existing models for auto-regressive language modeling. Our method comes with a low-cost $O(l)$ recurrent mode and an efficient $O(l \log_{2} l)$ parallel mode making use of highly optimized associative scan implementations. Furthermore, we derive an $O(l^2)$ surrogate attention mode, revealing remarkable implications for Transformer and recently proposed architectures. Specifically, we prove that our approach can be interpreted as providing data-controlled relative-positional information to Attention. While many existing models solely rely on data-controlled cumulative sums for context aggregation, our findings suggest that incorporating data-controlled complex cumulative products may be a crucial step towards more powerful sequence models.


Generalizations of R0 and SSM properties; Extended Horizontal Linear Complementarity Problem

Abstract: In this paper, we first introduce R0-W and SSM-W properties for the set of matrices which is a generalization of R0 and the strictly semimonotone matrix. We then prove some existence results for the extended horizontal linear complementarity problem when the involved matrices have these properties. With an additional condition on the set of matrices, we prove that the SSM-W property is equivalent to the unique solution for the corresponding extended horizontal linear complementarity problems. Finally, we give a necessary and sufficient condition for the connectedness of the solution set of the extended horizontal linear complementarity problems.


Space Weather: From Solar Origins to Risks and Hazards Evolving in Time

Abstract: Space Weather is the portion of space physics that has a direct effect on humankind. Space Weather is an old branch of space physics that originates back to 1808 with the publication of a paper by the great naturalist Alexander von Humboldt (von Humboldt, 1808). Space Weather is currently experiencing explosive growth, because its effects on human technologies have become more and more diverse. Space Weather is due to the variability of solar processes that cause interplanetary, magnetospheric, ionospheric, atmospheric and ground level effects. Space Weather can at times have strong impacts on technological systems and human health. The threats and risks are not hypothetical, and in the event of extreme Space Weather events the consequences could be quite severe for humankind. The purpose of the review is to give a brief overall view of the full chain of physical processes responsible for Space Weather risks and hazards, tracing them from solar origins to effects and impacts in interplanetary space, in the Earth's magnetosphere and ionosphere and at the ground. The paper shows that the risks associated with Space Weather have not been constant over time; they have evolved as our society becomes more and more technologically advanced. The paper begins with a brief introduction to the Carrington event. Next, the descriptions of the strongest known Space Weather processes are reviewed. The concepts of geomagnetic storms and substorms are briefly introduced. The main effects/impacts of Space Weather are also considered, including geomagnetically induced currents (GICs) which are thought to cause power outages. The effects of radiation on avionics and human health, ionospheric effects and impacts, and thermosphere effects and satellite drag will also be discussed. Finally, we will discuss the current challenges of Space Weather forecasting and examine some of the worst-case scenarios.

Frontiers in Astronomy and Space Sciences


Simplifying and Understanding State Space Models with Diagonal Linear RNNs

Abstract: Sequence models based on linear state spaces (SSMs) have recently emerged as a promising choice of architecture for modeling long range dependencies across various modalities. However, they invariably rely on discretization of a continuous state space, which complicates their presentation and understanding. In this work, we dispose of the discretization step, and propose a model based on vanilla Diagonal Linear RNNs ($\mathrm{DLR}$). We empirically show that, despite being conceptually much simpler, $\mathrm{DLR}$ is as performant as previously-proposed SSMs on a variety of tasks and benchmarks including Long Range Arena and raw speech classification. Moreover, we characterize the expressivity of SSMs (including $\mathrm{DLR}$) and attention-based models via a suite of $13$ synthetic sequence-to-sequence tasks involving interactions over tens of thousands of tokens, ranging from simple operations, such as shifting an input sequence, to detecting co-dependent visual features over long spatial ranges in flattened images. We find that while SSMs report near-perfect performance on tasks that can be modeled via $\textit{few}$ convolutional kernels, they struggle on tasks requiring $\textit{many}$ such kernels and especially when the desired sequence manipulation is $\textit{context-dependent}$. Despite these limitations, $\mathrm{DLR}$ reaches high performance on two higher-order reasoning tasks $\mathrm{ListOpsSubTrees}$ and $\mathrm{PathfinderSegmentation}\text{-}\mathrm{256}$ with input lengths $8K$ and $65K$ respectively, and gives encouraging performance on $\mathrm{PathfinderSegmentation}\text{-}\mathrm{512}$ with input length $262K$ for which attention is not a viable choice.


Space Weather Observations, Modeling, and Alerts in Support of Human Exploration of Mars

Abstract: Space weather observations and modeling at Mars have begun but they must be significantly increased to support the future of Human Exploration on the Red Planet. A comprehensive space weather understanding of a planet without a global magnetosphere and a thin atmosphere is very different from our situation at Earth so there is substantial fundamental research remaining. It is expected that the development of suitable models will lead to a comprehensive operational Mars space weather alert (MSWA) system that would provide rapid dissemination of information to Earth controllers, astronauts in transit, and those in the exploration zone (EZ) on the surface by producing alerts that are delivered rapidly and are actionable. To illustrate the importance of such a system, we use a magnetohydrodynamic code to model an extreme Carrington-type coronal mass ejection (CME) event at Mars. The results show a significant induced surface field of nearly 3000 nT on the dayside that could radically affect unprotected electrical systems that would dramatically impact human survival on Mars. Other associated problems include coronal mass ejection (CME) shock-driven acceleration of solar energetic particles producing large doses of ionizing radiation at the Martian surface. In summary, along with working more closely with international partners, the next Heliophysics Decadal Survey must include a new initiative to meet expected demands for space weather forecasting in support of humans living and working on the surface of Mars. It will require significant effort to coordinate NASA and the international community contributions.

James L. Green
Chuanfei Dong
Michael Hesse
C. Alex Young
Vladimir Airapetian


How Do Shock Waves Define the Space-Time Structure of Gradual Solar Energetic Particle Events?

Abstract: We revisit the full variety of observed temporal and spatial distributions of energetic solar protons in "gradual" solar energetic-particle (SEP) events resulting from the spatial variations in the shock waves that accelerate them. Differences in the shock strength at the solar longitude of a spacecraft and at the footpoint of its connecting magnetic field line, nominally 55 degrees to the west, drive much of that variation. The shock wave itself, together with energetic particles trapped near it by self-amplified Alfven waves, forms an underlying autonomous structure that can drive across magnetic field lines intact, spreading proton intensities in a widening SEP longitude distribution. During the formation of this fundamental structure, historically called an "energetic storm particle" (ESP) event, many SEPs leak away early, amplifying waves as they flow along well-connected field lines and broaden the distribution outward; behind this structure between the shock and the Sun a "reservoir" of quasi-trapped SEPs forms. Very large SEP events are complicated by additional extensive wave growth that can spread an extended ESP-like trapping region. The multiplicity of shock-related processes contributing to the observed SEP profiles causes correlations of the events to be poorly represented by the peak intensities commonly used. In fact, the extensive spatial distributions of SEPs are sometimes interwoven with the structures of the shocks that have accelerated them and sometimes free. We should consider new questions: Which extremes of the shock contribute most to the SEPs profile of an event, (1) the shock at the longitude of a spacecraft, (2) the shock ~55 degrees to the west at the footpoint of the field, or (3) SEPs that have collected in the reservoir? How does the space-time distribution of SEPs correspond with the underlying space-time distribution of shock strength?


Simplified State Space Layers for Sequence Modeling

Abstract: Models using structured state space sequence (S4) layers have achieved state-of-the-art performance on long-range sequence modeling tasks. An S4 layer combines linear state space models (SSMs), the HiPPO framework, and deep learning to achieve high performance. We build on the design of the S4 layer and introduce a new state space layer, the S5 layer. Whereas an S4 layer uses many independent single-input, single-output SSMs, the S5 layer uses one multi-input, multi-output SSM. We establish a connection between S5 and S4, and use this to develop the initialization and parameterization used by the S5 model. The result is a state space layer that can leverage efficient and widely implemented parallel scans, allowing S5 to match the computational efficiency of S4, while also achieving state-of-the-art performance on several long-range sequence modeling tasks. S5 averages 87.4% on the long range arena benchmark, and 98.5% on the most difficult Path-X task.

Jimmy T. H. Smith
Andrew Warrington
Scott W. Linderman

International Conference on Learning Representations


Hidden Parameter Recurrent State Space Models For Changing Dynamics Scenarios

Abstract: Recurrent State-space models (RSSMs) are highly expressive models for learning patterns in time series data and system identification. However, these models assume that the dynamics are fixed and unchanging, which is rarely the case in real-world scenarios. Many control applications often exhibit tasks with similar but not identical dynamics which can be modeled as a latent variable. We introduce the Hidden Parameter Recurrent State Space Models (HiP-RSSMs), a framework that parametrizes a family of related dynamical systems with a low-dimensional set of latent factors. We present a simple and effective way of learning and performing inference over this Gaussian graphical model that avoids approximations like variational inference. We show that HiP-RSSMs outperforms RSSMs and competing multi-task models on several challenging robotic benchmarks both on real-world systems and simulations.

Vaisakh Shaj
Dieter Buchler
Rohit Sonker
Philipp Becker
Gerhard Neumann


Achievements and Lessons Learned from Successful Small Satellite Missions for Space Weather-Oriented Research

Abstract: When the first CubeSats were launched nearly two decades ago, few people believed that the miniature satellites would likely prove to be a useful scientific tool. Skeptics abounded. However, the last decade has seen the highly successful implementation of space missions that make creative and innovative use of fast-advancing CubeSat and small satellite technology to carry out important science experiments and missions. Several projects now have used CubeSats to obtain first-of-their-kind observations and findings that have formed the basis for high-profile engineering and science publications, thereby establishing without doubt the scientific value and broad utility of CubeSats. In this paper, we describe recent achievements and lessons learned from a representative selection of successful CubeSat missions with a space weather focus. We conclude that these missions were successful in part because their limited resources promoted not only mission focus but also appropriate risk-taking for comparatively high science return. Quantitative analysis of refereed publications from these CubeSat missions and several larger missions reveals that mission outcome metrics compare favorably when publication number is normalized by mission cost or if expressed as a weighted net scientific impact of all mission publications.

Harlan E. Spence
Amir Caspi
Hasan Bahcivan
Jesus Nieves-Chinchilla
Geoff Crowley
James Cutler
Chad Fish
David Jackson
Therese Moretto Jørgensen
David Klumpar
Xinlin Li
James P. Mason
Nick Paschalidis
John Sample
Sonya Smith
Charles M. Swenson
Thomas N. Woods

Space Weather, Vol. 20, Issue 7, e2021SW003031 (21pp); 2022 June 28


Optical Atomic Clock aboard an Earth-orbiting Space Station (OACESS): Enhancing searches for physics beyond the standard model in space

Abstract: We present a concept for a high-precision optical atomic clock (OAC) operating on an Earth-orbiting space station. This pathfinder science mission will compare the space-based OAC with one or more ultra-stable terrestrial OACs to search for space-time-dependent signatures of dark scalar fields that manifest as anomalies in the relative frequencies of station-based and ground-based clocks. This opens the possibility of probing models of new physics that are inaccessible to purely ground-based OAC experiments where a dark scalar field may potentially be strongly screened near Earth's surface. This unique enhancement of sensitivity to potential dark matter candidates harnesses the potential of space-based OACs.

Vladimir Schkolnik
Dmitry Budker
Oliver Fartmann
Victor Flambaum
Leo Hollberg
Tigran Kalaydzhyan
Shimon Kolkowitz
Markus Krutzik
Andrew Ludlow
Nathan Newbury
Christoph Pyrlik
Laura Sinclair
Yevgeny Stadnik
Ingmari Tietje
Jun Ye
Jason Williams


The Geography of Facebook Groups in the United States

Abstract: We use exploratory factor analysis to investigate the online persistence of known community-level patterns of social capital variance in the U.S. context. Our analysis focuses on Facebook groups, specifically those that tend to connect users in the same local area. We investigate the relationship between established, localized measures of social capital at the county level and patterns of participation in Facebook groups in the same areas. We identify four main factors that distinguish Facebook group engagement by county. The first captures small, private groups, dense with friendship connections. The second captures very local and small groups. The third captures non-local, large, public groups, with more age mixing. The fourth captures partially local groups of medium to large size. The first and third factor correlate with community level social capital measures, while the second and fourth do not. Together and individually, the factors are predictive of offline social capital measures, even controlling for various demographic attributes of the counties. Our analysis reveals striking patterns of correlation between established measures of social capital and patterns of online interaction in local Facebook groups. To our knowledge this is the first systematic test of the association between offline regional social capital and patterns of online community engagement in the same regions.

Amaç Herdağdelen
Lada Adamic
Bogdan State

International Conference on Web and Social Media


Small Satellite Mission Concepts for Space Weather Research and as Pathfinders for Operations

Abstract: Recent advances in miniaturization and commercial availability of critical satellite subsystems and detector technology have made small satellites (SmallSats, including CubeSats) an attractive, low-cost potential solution for space weather research and operational needs. Motivated by the 1st International Workshop on SmallSats for Space Weather Research and Forecasting, held in Washington, DC on 1-4 August 2017, we discuss the need for advanced space weather measurement capabilities, driven by analyses from the World Meteorological Organization (WMO), and how SmallSats can efficiently fill these measurement gaps. We present some current, recent missions and proposed/upcoming mission concepts using SmallSats that enhance space weather research and provide prototyping pathways for future operational applications; how they relate to the WMO requirements; and what challenges remain to be overcome to meet the WMO goals and operational needs in the future. With additional investment from cognizant funding agencies worldwide, SmallSats -- including standalone missions and constellations -- could significantly enhance space weather research and, eventually, operations, by reducing costs and enabling new measurements not feasible from traditional, large, monolithic missions.

Amir Caspi
M. Barthelemy
C. D. Bussy-Virat
I. J. Cohen
C. E. DeForest
D. R. Jackson
A. Vourlidas
T. Nieves-Chinchilla

Space Weather, Vol. 20, Issue 2, e2020SW002554 (17pp); 2022 January 31


Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers

Abstract: Recurrent neural networks (RNNs), temporal convolutions, and neural differential equations (NDEs) are popular families of deep learning models for time-series data, each with unique strengths and tradeoffs in modeling power and computational efficiency. We introduce a simple sequence model inspired by control systems that generalizes these approaches while addressing their shortcomings. The Linear State-Space Layer (LSSL) maps a sequence $u \mapsto y$ by simply simulating a linear continuous-time state-space representation $\dot{x} = Ax + Bu, y = Cx + Du$. Theoretically, we show that LSSL models are closely related to the three aforementioned families of models and inherit their strengths. For example, they generalize convolutions to continuous-time, explain common RNN heuristics, and share features of NDEs such as time-scale adaptation. We then incorporate and generalize recent theory on continuous-time memorization to introduce a trainable subset of structured matrices $A$ that endow LSSLs with long-range memory. Empirically, stacking LSSL layers into a simple deep neural network obtains state-of-the-art results across time series benchmarks for long dependencies in sequential image classification, real-world healthcare regression tasks, and speech. On a difficult speech classification task with length-16000 sequences, LSSL outperforms prior approaches by 24 accuracy points, and even outperforms baselines that use hand-crafted features on 100x shorter sequences.

Albert Gu
Isys Johnson
Karan Goel
Khaled Saab
Tri Dao
Atri Rudra
Christopher Ré


RTSNet: Learning to Smooth in Partially Known State-Space Models (Preprint)

Abstract: The smoothing task is core to many signal processing applications. A widely popular smoother is the Rauch-Tung-Striebel (RTS) algorithm, which achieves minimal mean-squared error recovery with low complexity for linear Gaussian state space (SS) models, yet is limited in systems that are only partially known, as well as non-linear and non-Gaussian. In this work we propose RTSNet, a highly efficient model-based and data-driven smoothing algorithm suitable for partially known SS models. RTSNet integrates dedicated trainable models into the flow of the classical RTS smoother, while iteratively refining its sequence estimate via deep unfolding methodology. As a result, RTSNet learns from data to reliably smooth when operating under model mismatch and non-linearities while retaining the efficiency and interpretability of the traditional RTS smoothing algorithm. Our empirical study demonstrates that RTSNet overcomes non-linearities and model mismatch, outperforming classic smoothers operating with both mismatched and accurate domain knowledge. Moreover, while RTSNet is based on compact neural networks, which leads to faster training and inference times, it demonstrates improved performance over previously proposed deep smoothers in non-linear settings.

Guy Revach
Xiaoyong Ni
Nir Shlezinger
Ruud J. G. van Sloun
Yonina C. Eldar


OSPREI: A Coupled Approach to Modeling CME-Driven Space Weather with Automatically-Generated, User-Friendly Outputs

Abstract: Coronal Mass Ejections (CMEs) drive space weather activity at Earth and throughout the solar system. Current CME-related space weather predictions rely on information reconstructed from coronagraphs, sometimes from only a single viewpoint, to drive a simple interplanetary propagation model, which only gives the arrival time or limited additional information. We present the coupling of three established models into OSPREI (Open Solar Physics Rapid Ensemble Information), a new tool that describes Sun-to-Earth CME behavior, including the location, orientation, size, shape, speed, arrival time, and internal thermal and magnetic properties, on the timescale needed for forecasts. First, ForeCAT describes the trajectory that a CME takes through the solar corona. Second, ANTEATR simulates the propagation, including expansion and deformation, of a CME in interplanetary space and determines the evolution of internal properties via conservation laws. Finally, FIDO produces in situ profiles for a CME's interaction with a synthetic spacecraft. OSPREI includes ensemble modeling by varying each input parameter to probe any uncertainty in their values, yielding probabilities for all outputs. Standardized visualizations are automatically generated, providing easily-accessible, essential information for space weather forecasting. We show OSPREI results for CMEs observed in the corona on 2021 April 22 and 2021 May 09. We approach these CME as a forecasting proof-of-concept, using information analogous to what would be available in real time rather than fine-tuning input parameters to achieve a best fit for a detailed scientific study. The OSPREI prediction shows good agreement with the arrival time and in situ properties.


Quantum Metric and Correlated States in Two-dimensional Systems

Abstract: The recent realization of twisted, two-dimensional, bilayers exhibiting strongly correlated states has created a platform in which the relation between the properties of the electronic bands and the nature of the correlated states can be studied in unprecedented ways. The reason is that these systems allow extraordinary control of the electronic bands' properties, for example by varying the relative twist angle between the layers forming the system. In particular, in twisted bilayers the low energy bands can be tuned to be very flat and with a nontrivial quantum metric. This allows the quantitative and experimental exploration of the relation between the metric of Bloch quantum states and the properties of correlated states. In this work we first review the general connection between quantum metric and the properties of correlated states that break a continuous symmetry. We then discuss the specific case when the correlated state is a superfluid and show how the quantum metric is related to the superfluid stiffness. To exemplify such relation we show results for the case of superconductivity in magic angle twisted bilayer graphene. We conclude by discussing possible research directions to further elucidate the connection between quantum metric and correlated states' properties.

Current opinion in solid state & materials science


Elysium: Context-Aware Bytecode-Level Patching to Automatically Heal Vulnerable Smart Contracts

Abstract: Fixing bugs is easiest by patching source code. However, source code is not always available: only 0.3% of the ~49M smart contracts that are currently deployed on Ethereum have their source code publicly available. Moreover, since contracts may call functions from other contracts, security flaws in closed-source contracts may affect open-source contracts as well. However, current state-of-the-art approaches that operate on closed-source contracts (i.e., EVM bytecode), such as EVMPatch and SmartShield, make use of purely hard-coded templates that leverage fix patching patterns. As a result, they cannot dynamically adapt to the bytecode that is being patched, which severely limits their flexibility and scalability. For instance, when patching integer overflows using hard-coded templates, a particular patch template needs to be employed as the bounds to be checked are different for each integer size. In this paper, we propose Elysium, a scalable approach towards automatic smart contract repair at the bytecode level. Elysium combines template-based and semantic-based patching by inferring context information from bytecode. Elysium is currently able to patch 7 different types of vulnerabilities in smart contracts automatically and can easily be extended with new templates and new bug-finding tools. We evaluate its effectiveness and correctness using 3 different datasets by replaying more than 500K transactions on patched contracts. We find that Elysium outperforms existing tools by patching at least 30% more contracts correctly. Finally, we also compare the overhead of Elysium in terms of deployment and transaction cost. In comparison to other tools, we find that generally Elysium minimizes the runtime cost (i.e., transaction cost) up to a factor of 1.7, for only a marginally higher deployment cost, where deployment cost is a one-time cost as compared to the runtime cost.

Christof Ferreira Torres
Hugo Jonker
Radu State

International Symposium on Recent Advances in Intrusion Detection


What Sustained Multi-Disciplinary Research Can Achieve: The Space Weather Modeling Framework

Abstract: MHD-based global space weather models have mostly been developed and maintained at academic institutions. While the "free spirit" approach of academia enables the rapid emergence and testing of new ideas and methods, the lack of long-term stability and support makes this arrangement very challenging. This paper describes a successful example of a university-based group, the Center of Space Environment Modeling (CSEM) at the University of Michigan, that developed and maintained the Space Weather Modeling Framework (SWMF) and its core element, the BATS-R-US extended MHD code. It took a quarter of a century to develop this capability and reach its present level of maturity that makes it suitable for research use by the space physics community through the Community Coordinated Modeling Center (CCMC) as well as operational use by the NOAA Space Weather Prediction Center (SWPC).

Tamas I. Gombosi
Yuxi Chen
Alex Glocer
Zhenguang Huang
Xianzhe Jia
Michael W. Liemohn
Ward B. Manchester
Tuija Pulkkinen
Nishtha Sachdeva
Qusai Al Shidi
Igor V. Sokolov
Judit Szente
Valeriy Tenishev
Gabor Toth
Bart van der Holst
Daniel T. Welling
Lulu Zhao
Shasha Zou

Journal of Space Weather and Space Climate


New cosmic ray observations at Syowa Station in the Antarctic for space weather study

Abstract: Muon detectors and neutron monitors were recently installed at Syowa Station, in the Antarctic, to observe different types of secondary particles resulting from cosmic ray interactions simultaneously from the same location. Continuing observations will give new insight into the response of muon detectors to atmospheric and geomagnetic effects. Operation began in February, 2018 and the system has been stable with a duty-cycle exceeding 94%. Muon data shows a clear seasonal variation, which is expected from the atmospheric temperature effect. We verified successful operation by showing that the muon and neutron data are consistent with those from other locations by comparing intensity variations during a space weather event. We have established a web page to make real time data available with interactive graphics (http://polaris.nipr.ac.jp/~cosmicrays/).

C. Kato
W. Kihara
Y. Ko
A. Kadokura
R. Kataoka
P. Evenson
S. Uchida
S. Kaimi
Y. Nakamura
H. A. Uchida
K. Murase
K. Munakata


On the Scaling and Spacing of Extra-Solar Multi-Planet Systems

Abstract: We investigate whether certain extra-solar multi-planet systems simultaneously follow the scaling and spacing rules of the angular-momentum-deficit model. The masses and semi-major axes of exoplanets in ten multi-planet systems are considered. It is found that GJ 667C, HD 215152, HD 40307, and Kepler-79 systems are currently close to configurations of the angular-momentum-deficit model. In a gas-poor scenario, GJ 3293, HD 141399, and HD 34445 systems are those which had a configuration of the angular-momentum-deficit model in the past and get scattered away due to post gaseous effects. In addition, no matter in gas-free or gas-poor scenario, 55 Cnc, GJ 876, and WASP-47 systems do not follow the angular-momentum-deficit model. Therefore, our results reveal important formation histories of these multi-planet systems.

Li-Chin Yeh
Ing-Guey Jiang
Sridhar Gajendran


Addressing Gaps in Space Weather Operations and Understanding with Small Satellites

Abstract: Gaps in space weather observations that can be addressed with small satellites are identified. Potential improvements in solar inputs to space weather models, space radiation control, estimations of energy budget of the upper Earth's atmosphere, and satellite drag modeling are briefly discussed. Key observables, instruments and observation strategies by small satellites are recommended. Tracking optimization for small satellites is proposed.

Olga Verkhoglyadova
Charles Bussy-Virat
Amir Caspi
David Jackson
Vladimir Kalegaev
Jeffrey Klenzing
Jesus Nieves-Chinchilla
Angelos Vourlidas

Environmental Science
Computer Science
Physics

Space Weather, Vol. 19, Issue 3, e2020SW002566 (7pp); 2021 March 23


Toward a Next Generation Particle Precipitation Model: Mesoscale Prediction Through Machine Learning (a Case Study and Framework for Progress)

Abstract: We advance the modeling capability of electron particle precipitation from the magnetosphere to the ionosphere through a new database and use of machine learning (ML) tools to gain utility from those data. We have compiled, curated, analyzed, and made available a new and more capable database of particle precipitation data that includes 51 satellite years of Defense Meteorological Satellite Program (DMSP) observations temporally aligned with solar wind and geomagnetic activity data. The new total electron energy flux particle precipitation nowcast model, a neural network called PrecipNet, takes advantage of increased expressive power afforded by ML approaches to appropriately utilize diverse information from the solar wind and geomagnetic activity and, importantly, their time histories. With a more capable representation of the organizing parameters and the target electron energy flux observations, PrecipNet achieves a >50% reduction in errors from a current state-of-the-art model oval variation, assessment, tracking, intensity, and online nowcasting (OVATION Prime), better captures the dynamic changes of the auroral flux, and provides evidence that it can capably reconstruct mesoscale phenomena. We create and apply a new framework for space weather model evaluation that culminates previous guidance from across the solar-terrestrial research community. The research approach and results are representative of the "new frontier" of space weather research at the intersection of traditional and data science-driven discovery and provides a foundation for future efforts.

Ryan M. McGranaghan
Jack Ziegler
Téo Bloch
Spencer Hatch
Enrico Camporeale
Kristina Lynch
Mathew Owens
Jesper Gjerloev
Binzheng Zhang
Susan Skone

Physics
Mathematics
Environmental Science


Bayesian recurrent state space model for rs-fMRI

Abstract: We propose a hierarchical Bayesian recurrent state space model for modeling switching network connectivity in resting state fMRI data. Our model allows us to uncover shared network patterns across disease conditions. We evaluate our method on the ADNI2 dataset by inferring latent state patterns corresponding to altered neural circuits in individuals with Mild Cognitive Impairment (MCI). In addition to states shared across healthy and individuals with MCI, we discover latent states that are predominantly observed in individuals with MCI. Our model outperforms current state of the art deep learning method on ADNI2 dataset.

Arunesh Mittal
Scott Linderman
John Paisley
Paul Sajda


International Coordination and Support for SmallSat-enabled Space Weather Activities

Abstract: Advances in space weather science and small satellite (SmallSat) technology have proceeded in parallel over the past two decades, but better communication and coordination is needed among the respective worldwide communities contributing to this rapid progress. We identify six areas where improved international coordination is especially desirable, including: (1) orbital debris mitigation; (2) spectrum management; (3) export control regulations; (4) access to timely and low-cost launch opportunities; (5) inclusive data policies; and (6) education. We argue the need for internationally coordinated policies and programs to promote the use of SmallSats for space weather research and forecasting while realizing maximum scientific and technical advances through the integration of these two increasingly important endeavors.

Teresa Nieves-Chinchilla
Bhavya Lal
Robert Robinson
Amir Caspi
David R. Jackson
Therese Moretto Jørgensen
James Spann

Space Weather, Vol. 18, Issue 12, e2020SW002568 (6pp); 2020 November 25


Domain of Influence analysis: implications for Data Assimilation in space weather forecasting

Abstract: Solar activity, ranging from the background solar wind to energetic coronal mass ejections (CMEs), is the main driver of the conditions in the interplanetary space and in the terrestrial space environment, known as space weather. A better understanding of the Sun-Earth connection carries enormous potential to mitigate negative space weather effects with economic and social benefits. Effective space weather forecasting relies on data and models. In this paper, we discuss some of the most used space weather models, and propose suitable locations for data gathering with space weather purposes. We report on the application of \textit{Representer analysis (RA)} and \textit{Domain of Influence (DOI) analysis} to three models simulating different stages of the Sun-Earth connection: the OpenGGCM and Tsyganenko models, focusing on solar wind - magnetosphere interaction, and the PLUTO model, used to simulate CME propagation in interplanetary space. Our analysis is promising for space weather purposes for several reasons. First, we obtain quantitative information about the most useful locations of observation points, such as solar wind monitors. For example, we find that the absolute values of the DOI are extremely low in the magnetospheric plasma sheet. Since knowledge of that particular sub-system is crucial for space weather, enhanced monitoring of the region would be most beneficial. Second, we are able to better characterize the models. Although the current analysis focuses on spatial rather than temporal correlations, we find that time-independent models are less useful for Data Assimilation activities than time-dependent models. Third, we take the first steps towards the ambitious goal of identifying the most relevant heliospheric parameters for modelling CME propagation in the heliosphere, their arrival time, and their geoeffectiveness at Earth.

Dimitrios Millas
Maria Elena Innocenti
Brecht Laperre
Joachim Raeder
Stefaan Poedts
Giovanni Lapenta


Archaeology in a Vacuum: Obstacles to and Solutions for Developing a Real Space Archaeology

Abstract: This paper outlines some of the difficulties faced by archaeologists studying human activity in outer space. The International Space Station Archaeological Project has identified solutions to these problems, including the use of historic photographic archives and documentation of discard practices such as processes associated with the return of space-flown items to Earth.


Searching for Supersymmetry: The $μν$SSM

Abstract: We review the role played by the '$μ$ from $ν$' supersymmetric standard model ($μν$SSM) in the search for supersymmetry. First, we discuss its theoretical motivation, that is the simultaneous solution of $μ$- and $ν$-problems through the introduction of right-handed neutrinos. The latter produces $R$-parity violation (RPV), giving rise to interesting signals of new physics. As by-products, in the $μν$SSM there are dark matter candidates, and electroweak baryogenesis can be realized. Then, we survey signals by which the model could be tested at the large hadron collider (LHC). In addition to the enlarged Higgs sector with sneutrinos, we put special emphasis in analyzing the intimate connection between the lightest supersymmetric particle (LSP) lifetime and the size of neutrino Yukawa couplings. Displaced vertices and/or multileptons are some of the interesting signatures that can be probed. Finally, we discuss possible extensions of the $μν$SSM such as the inclusion in the superpotential of the conventional trilinear lepton-number violating couplings, the addition of an extra $U(1)'$ gauge group to the symmetry of the standard model, or the reinterpretation of the Higgs doublets as a fourth family of leptons superfields motivating the existence of a fourth family of vector-like quark doublet superfields.

The European Physical Journal Special Topics