From these results, we find that IBS yields a consistent but modest improvement on the Panoptic Quality (PQ) metric #REFR .

We apply our Intra-Batch Supervision (IBS) to two stateof-the-art unified panoptic segmentation methods, and evaluate the performance on two high-resolution datasets in Table 2 .

This is mainly caused by a boost in the PQ for things classes, PQ th , which consistently shows considerable improvements.

This is expected, because the aim of our IBS is to improve the confusion problem that occurs for thing segments.

As explained in Section 3.3, however, the ability of the PQ metric to capture errors for large objects of frequently-occuring classes, such as the confusion problem that we are addressing, is limited.

To better capture this, we also report on the pixel accuracy (Acc th ) and pixel precision (Prec th ) for things classes.

For these metrics, we observe much more significant improvements, e.g., a +5.8 improvement for both metrics for Panoptic FCN on Cityscapes, and +4.0 and +5.3 for Mask2Former on Mapillary Vistas.

Intra-Batch Supervision For Panoptic Segmentation On High-Resolution Images

Abstract: Unified panoptic segmentation methods are achieving state-of-the-art results on several datasets. To achieve these results on high-resolution datasets, these methods apply crop-based training. In this work, we find that, although crop-based training is advantageous in general, it also has a harmful side-effect. Specifically, it limits the ability of unified networks to discriminate between large object instances, causing them to make predictions that are confused between multiple instances. To solve this, we propose Intra-Batch Supervision (IBS), which improves a network's ability to discriminate between instances by introducing additional supervision using multiple images from the same batch. We show that, with our IBS, we successfully address the confusion problem and consistently improve the performance of unified networks. For the high-resolution Cityscapes and Mapillary Vistas datasets, we achieve improvements of up to +2.5 on the Panoptic Quality for thing classes, and even more considerable gains of up to +5.8 on both the pixel accuracy and pixel precision, which we identify as better metrics to capture the confusion problem. PQ th Acc th 51.1 85.1 PQ th Acc th 52.2 81.3 PQ th Acc th 54.7 87.1 (a) Full-image training (b) Crop-based training (c) Crop-based training + IBS (ours)

Panoptic Segmentation

Abstract: We propose and study a task we name panoptic segmentation (PS). Panoptic segmentation unifies the typically distinct tasks of semantic segmentation (assign a class label to each pixel) and instance segmentation (detect and segment each object instance). The proposed task requires generating a coherent scene segmentation that is rich and complete, an important step toward real-world vision systems. While early work in computer vision addressed related image/scene parsing tasks, these are not currently popular, possibly due to lack of appropriate metrics or associated recognition challenges. To address this, we propose a novel panoptic quality (PQ) metric that captures performance for all classes (stuff and things) in an interpretable and unified manner. Using the proposed metric, we perform a rigorous study of both human and machine performance for PS on three existing datasets, revealing interesting insights about the task. The aim of our work is to revive the interest of the community in a more unified view of image segmentation. For more analysis and up-to-date results, please check the arXiv version of the paper: {\small\url{https://arxiv.org/abs/1801.00868}}.


However, as shown in #REFR , such requirements are unrealistic on the widely-used denoisers mentioned above, as they do not have symmetric Jacobians.

PnP versions of proximal algorithms have been used to solve image restoration problems such as for example PnP-PGD in #OTHEREFR , PnP-ADMM and PnP-DRS in #OTHEREFR and PnP-HQS in #OTHEREFR .

In #OTHEREFR an explicit regularisation by denoising (RED) strategy was designed in terms of an explicit function R(•) defined, for generic image denoiser D, by:

Under conditions of local homogeneity, non-expansiveness, and Jacobian symmetry, D was shown to be indeed equivalent to a gradient step on R #OTHEREFR , that is,

In order to overcome this limitation, in #OTHEREFR , the authors proposed to formulate, similar to RED, a gradient step denoiser of the form:

where R σ : R n 2 → R is a scalar function parameterised by a neural network

Interestingly, under mild structural assumption on D σ , the authors are able to prove sound convergence guarantees for the underlying nonconvex optimisation problem defined in terms of a non-trivial (but explicit) regularisation function R(•).

Fluctuation-Based Deconvolution In Fluorescence Microscopy Using Plug-And-Play Denoisers

Abstract: The spatial resolution of images of living samples obtained by fluorescence microscopes is physically limited due to the diffraction of visible light, which makes the study of entities of size less than the diffraction barrier (around 200 nm in the x-y plane) very challenging. To overcome this limitation, several deconvolution and super-resolution techniques have been proposed. Within the framework of inverse problems, modern approaches in fluorescence microscopy reconstruct a superresolved image from a temporal stack of frames by carefully designing suitable hand-crafted sparsity-promoting regularisers. Numerically, such approaches are solved by proximal gradient-based iterative schemes. Aiming at obtaining a reconstruction more adapted to sample geometries (e.g. thin filaments), we adopt a plug-and-play denoising approach with convergence guarantees and replace the proximity operator associated with the explicit image regulariser with an image denoiser (i.e. a pretrained network) which, upon appropriate training, mimics the action of an implicit prior. To account for the independence of the fluctuations between molecules, the model relies on second-order statistics. The denoiser is then trained on covariance images coming from data representing sequences of fluctuating fluorescent molecules with filament structure. The method is evaluated on both simulated and real fluorescence microscopy images, showing its ability to correctly reconstruct filament structures with high values of peak signal-to-noise ratio (PSNR).

Regularization By Denoising: Clarifications And New Interpretations

Abstract: Regularization by denoising (RED), as recently proposed by Romano, Elad, and Milanfar, is powerful image-recovery framework that aims to minimize an explicit regularization objective constructed from a plug-in image-denoising function. Experimental evidence suggests that the RED algorithms are a state of the art. We claim, however, that explicit regularization does not explain the RED algorithms. In particular, we show that many of the expressions in the paper by Romano et al. hold only when the denoiser has a symmetric Jacobian, and we demonstrate that such symmetry does not occur with practical denoisers such as nonlocal means, BM3D, TNRD, and DnCNN. To explain the RED algorithms, we propose a new framework called Score-Matching by Denoising (SMD), which aims to match a “score” (i.e., the gradient of a log-prior). We then show tight connections between SMD, kernel density estimation, and constrained minimum mean-squared error denoising. Furthermore, we interpret the RED algorithms from Romano et al. and propose new algorithms with acceleration and convergence guarantees. Finally, we show that the RED algorithms seek a consensus equilibrium solution, which facilitates a comparison to plug-and-play ADMM.


In this paper, we mainly focus on the Transformer-Transducer (T-T) architecture #REFR and investigate cross-modality learning methods to leverage text-only data for improving the performance of the Mandarin-English code-switching ASR system.

Selfsupervised pretraining with multi-lingual unlabeled data without code-switching speech has also been shown effective to improve the performance of Mandarin-English code-switching ASR #OTHEREFR .

Transducer-based ASR model is very attractive in the industry since it provides a natural way for streaming.

However, it is not so straightforward to utilize unspoken text data except converting to synthesized speech or the features from TTS with a high computational cost #OTHEREFR .

There is no attention mechanism between the encoder and the decoder.

Although the prediction module plays as a Language Model (LM), it is different from the conventional LM due to the blank token #OTHEREFR that makes it harder to leverage external LM.

Code-Switching Text Generation And Injection In Mandarin-English Asr

Abstract: Code-switching speech refers to a means of expression by mixing two or more languages within a single utterance. Automatic Speech Recognition (ASR) with End-to-End (E2E) modeling for such speech can be a challenging task due to the lack of data. In this study, we investigate text generation and injection for improving the performance of an industry commonly-used streaming model, Transformer-Transducer (T-T), in Mandarin-English code-switching speech recognition. We first propose a strategy to generate codeswitching text data and then investigate injecting generated text into T-T model explicitly by Text-To-Speech (TTS) conversion or implicitly by tying speech and text latent spaces. Experimental results on the T-T model trained with a dataset containing 1,800 hours of real Mandarin-English code-switched speech show that our approaches to inject generated code-switching text significantly boost the performance of T-T models, i.e., 16% relative Token-based Error Rate (TER) reduction averaged on three evaluation sets, and the approach of tying speech and text latent spaces is superior to that of TTS conversion on the evaluation set which contains more homogeneous data with the training set.

Transformer Transducer: A Streamable Speech Recognition Model With Transformer Encoders And Rnn-T Loss

Abstract: In this paper we present an end-to-end speech recognition model with Transformer encoders that can be used in a streaming speech recognition system. Transformer computation blocks based on self-attention are used to encode both audio and label sequences independently. The activations from both audio and label encoders are combined with a feed-forward layer to compute a probability distribution over the label space for every combination of acoustic frame position and label history. This is similar to the Recurrent Neural Network Transducer (RNN-T) model, which uses RNNs for information encoding instead of Transformer encoders. The model is trained with the RNN-T loss well-suited to streaming decoding. We present results on the LibriSpeech dataset showing that limiting the left context for self-attention in the Transformer layers makes decoding computationally tractable for streaming, with only a slight degradation in accuracy. We also show that the full attention version of our model beats the-state-of-the art accuracy on the LibriSpeech benchmarks. Our results also show that we can bridge the gap between full attention and limited attention versions of our model by attending to a limited number of future frames.


The effect of recurrent time is similar to DuDoRNet #REFR and results are included in supplementary materials.

The reason for this design is that we weigh the utility of image reconstruction network and the synergy between image and K-space reconstruction networks over K-space reconstruction.

For K-GLIM and I-LDE, we try applying it to the other domain, both leading to performance drops. This demonstrates our design is domain-specific.

For X-TL, θ x is set to 1 by default, corresponding to the best choice for θ i .

As a result, only θ k is modified in the last row.

By gradually changing DuDoRNet to ours, our method also provides a possible hybridizing strategy for current CNN models.

Dudornext: A Hybrid Model For Dual-Domain Undersampled Mri Reconstruction

Abstract: Undersampled MRI reconstruction is crucial for accelerating clinical scanning procedures. Recent deep learning methods for MRI reconstruction adopt CNN or ViT as backbone, which lack in utilizing the complementary properties of CNN and ViT. In this paper, we propose DuDoRNeXt, whose backbone hybridizes CNN and ViT in an domainspecific, intra-stage way. Besides our hybrid vertical layout design, we introduce domain-specific modules for dual-domain reconstruction, namely image-domain parallel local detail enhancement and k-space global initialization. We evaluate different conventions of MRI reconstruction including image-domain, k-space-domain, and dual-domain reconstruction with a reference protocol on the IXI dataset and an in-house multicontrast dataset. DuDoRNeXt achieves significant improvements over competing deep learning methods.

Dudornet: Learning A Dual-Domain Recurrent Network For Fast Mri Reconstruction With Deep T1 Prior

Abstract: MRI with multiple protocols is commonly used for diagnosis, but it suffers from a long acquisition time, which yields the image quality vulnerable to say motion artifacts. To accelerate, various methods have been proposed to reconstruct full images from undersampled k-space data. However, these algorithms are inadequate for two main reasons. Firstly, aliasing artifacts generated in the image domain are structural and non-local, so that sole image domain restoration is insufficient. Secondly, though MRI comprises multiple protocols during one exam, almost all previous studies only employ the reconstruction of an individual protocol using a highly distorted undersampled image as input, leaving the use of fully-sampled short protocol (say T1) as complementary information highly underexplored. In this work, we address the above two limitations by proposing a Dual Domain Recurrent Network (DuDoRNet) with deep T1 prior embedded to simultaneously recover k-space and images for accelerating the acquisition of MRI with a long imaging protocol. Specifically, a Dilated Residual Dense Network (DRDNet) is customized for dual domain restorations from undersampled MRI data. Extensive experiments on different sampling patterns and acceleration rates demonstrate that our method consistently outperforms state-of-the-art methods, and can achieve SSIM up to 0.99 at $6 \times$ acceleration.


Anomaly Detection Approaches For Univariate Time Series

RePAD #REFR is an online real-time lightweight unsupervised time series anomaly detection approaches based on LSTM and the Look-Back and Predict-Forward strategy.

Greenhouse #OTHEREFR ) is a time series anomaly detection algorithm based on Long Short-Term Memory (LSTM), which is a special recurrent neural network suitable for long-term dependent tasks #OTHEREFR .

Greenhouse adopts a Look-Back and Predict-Forward strategy to learn the distribution of the training data.

For a given time point, a window of most recently observed data point values are used to predict future data point values.

However, Greenhouse is not an online approach since its LSTM model is trained with a pre-collected training data.

Besides, it requires users to determine a proper detection threshold.

RePAD utilizes a simple LSTM network (with only one hidden layer and ten hidden units) to train a LSTM model with shortterm historical data points, predict each upcoming data point, and then decide if each data point is anomalous based on a dynamically calculated detection threshold.

Different from Greenhouse, RePAD does not need to go through any offline training.

Instead, RePAD trains its LSTM model on the fly.

RePAD will keep using the same LSTM model if the model predicts well.

When the prediction error of the model is higher than or equal to a dynamically calculated detection threshold, RePAD will retrain another new model with recent data points.

Impact Of Deep Learning Libraries On Online Adaptive Lightweight Time Series Anomaly Detection

Abstract: Providing online adaptive lightweight time series anomaly detection without human intervention and domain knowledge is highly valuable. Several such anomaly detection approaches have been introduced in the past years, but all of them were only implemented in one deep learning library. With the development of deep learning libraries, it is unclear how different deep learning libraries impact these anomaly detection approaches since there is no such evaluation available. Randomly choosing a deep learning library to implement an anomaly detection approach might not be able to show the true performance of the approach. It might also mislead users in believing one approach is better than another. Therefore, in this paper, we investigate the impact of deep learning libraries on online adaptive lightweight time series anomaly detection by implementing two state-of-the-art anomaly detection approaches in three well-known deep learning libraries and evaluating how these two approaches are individually affected by the three deep learning libraries. A series of experiments based on four real-world open-source time series datasets were conducted. The results provide a good reference to select an appropriate deep learning library for online adaptive lightweight anomaly detection. ter decisions (Kieu et al.

Repad: Real-Time Proactive Anomaly Detection For Time Series

Abstract: During the past decade, many anomaly detection approaches have been introduced in different fields such as network monitoring, fraud detection, and intrusion detection. However, they require understanding of data pattern and often need a long off-line period to build a model or network for the target data. Providing real-time and proactive anomaly detection for streaming time series without human intervention and domain knowledge is highly valuable since it greatly reduces human effort and enables appropriate countermeasures to be undertaken before a disastrous damage, failure, or other harmful event occurs. However, this issue has not been well studied yet. To address it, this paper proposes RePAD, which is a Real-time Proactive Anomaly Detection algorithm for streaming time series based on unsupervised Long Short-Term Memory (LSTM). RePAD utilizes short-term historic data points to predict and determine whether or not the upcoming data point is a sign that an anomaly is likely to happen in the near future. By dynamically adjusting the detection threshold over time, RePAD is able to tolerate minor pattern change in time series and detect anomalies either proactively or on time. Experiments based on two time series datasets collected from the Numenta Anomaly Benchmark demonstrate that RePAD is able to proactively detect anomalies and provide early warnings in real time without human intervention and domain knowledge.


Although there is no location information in a graph, we can still locate the most (least) important area in a graph, like in an image, by using some explanation techniques #REFR .

Moreover, based on the improvement of the explanation techniques in the graph domain, #OTHEREFR proposed injecting the trigger into the most important or least important area of the sample.

However, that work does not provide any experimental analysis to confirm the assumptions made.

Also, there is no work so far on using explanation tools to explain the backdoor attack behavior in the graph domain. This work first raises a core question:

What is the attack performance when injecting trigger into the most or least important area of the sample?

To answer this question, we explore the impacts of the backdoor trigger-injecting position from the perspective of the most (MIAS) or least important area of the sample (LIAS).

As shown in experiments, we demonstrate that the attack performance of LIAS is better, where the difference from MIAS can even be significant. This observation inspires one further question:

Can we explain this difference? There are already some works on explaining backdoor attacks in the image domain through visualization techniques #OTHEREFR , #OTHEREFR .

For example, #OTHEREFR plotted the average activations of the backdoored model's last convolutional layer over clean and backdoored images to explain their attack.

#OTHEREFR used the Grad-CAM #OTHEREFR visualization method to explain the backdoor attack in federated learning.

One example of explaining a backdoor attack in the image domain with Grad-CAM is shown in Fig. 1 .

Rethinking The Trigger-Injecting Position In Graph Backdoor Attack

Abstract: Backdoor attacks have been demonstrated as a security threat for machine learning models. Traditional backdoor attacks intend to inject backdoor functionality into the model such that the backdoored model will perform abnormally on inputs with predefined backdoor triggers and still retain state-ofthe-art performance on the clean inputs. While there are already some works on backdoor attacks on Graph Neural Networks (GNNs), the backdoor trigger in the graph domain is mostly injected into random positions of the sample. There is no work analyzing and explaining the backdoor attack performance when injecting triggers into the most important or least important area in the sample, which we refer to as trigger-injecting strategies MIAS and LIAS, respectively. Our results show that, generally, LIAS performs better, and the differences between the LIAS and MIAS performance can be significant. Furthermore, we explain these two strategies' similar (better) attack performance through explanation techniques, which results in a further understanding of backdoor attacks in GNNs.

Gnnexplainer: Generating Explanations For Graph Neural Networks

Abstract: Graph Neural Networks (GNNs) are a powerful tool for machine learning on graphs. GNNs combine node feature information with the graph structure by recursively passing neural messages along edges of the input graph. However, incorporating both graph structure and feature information leads to complex models and explaining predictions made by GNNs remains unsolved. Here we propose GnnExplainer, the first general, model-agnostic approach for providing interpretable explanations for predictions of any GNN-based model on any graph-based machine learning task. Given an instance, GnnExplainer identifies a compact subgraph structure and a small subset of node features that have a crucial role in GNN's prediction. Further, GnnExplainer can generate consistent and concise explanations for an entire class of instances. We formulate GnnExplainer as an optimization task that maximizes the mutual information between a GNN's prediction and distribution of possible subgraph structures. Experiments on synthetic and real-world graphs show that our approach can identify important graph structures as well as node features, and outperforms alternative baseline approaches by up to 43.0% in explanation accuracy. GnnExplainer provides a variety of benefits, from the ability to visualize semantically relevant structures to interpretability, to giving insights into errors of faulty GNNs.


Here we followed the recent literature #REFR , where one layer and no more than 50 neurons are usually required to find the missing term in the differential equation.

The isotherm calculation one of the nodes in the computational graph -the one located leftmost layer.

This variable is then concatenated with the adsorbed amount and used as an input of the ANN that predicts the instantaneous uptake rate.

The structure of the ANN is usually chosen before training it.

In traditional deep learning, hyperparameter optimization is usually used to select the best architecture for a problem.

However, this is rarely explored in hybrid modeling as the training procedure is significantly more computationally demanding than in traditional deep learning.

In the present work, one-layer ANNs with hyperbolic tangent activation were used for all cases and a varying number of neurons between 15 and 25 with grid search.

Learning rates were set to 0.05 with Adaptive moment estimation (ADAM) optimizer #OTHEREFR and exponential learning rate decay every 20 iterations and 0.985 drop factor over 180 iterations.

After the second fit with ADAM, the quasi-newton Broyden-Fletcher-Goldfarb-Shanno (BFGS) method #OTHEREFR was employed until convergence.

Efficient Hybrid Modeling And Sorption Model Discovery For Non-Linear Advection-Diffusion-Sorption Systems: A Systematic Scientific Machine Learning Approach

Abstract: This study presents a systematic machine learning approach for creating efficient hybrid models and discovering sorption uptake models in non-linear advection-diffusion-sorption systems. It demonstrates an effective method to train these complex systems using gradientbased optimizers, adjoint sensitivity analysis, and JIT-compiled vector Jacobian products, combined with spatial discretization and adaptive integrators. Sparse and symbolic regression were employed to identify missing functions in the artificial neural network. The robustness of the proposed method was tested on an in-silico data set of noisy breakthrough curve observations of fixed-bed adsorption, resulting in a well-fitted hybrid model. The study successfully reconstructed sorption uptake kinetics using sparse and symbolic regression, and accurately predicted breakthrough curves using identified polynomials, highlighting the potential of the proposed framework for discovering sorption kinetic law structures.

Universal Differential Equations For Scientific Machine Learning

Abstract: In the context of science, the well-known adage "a picture is worth a thousand words" might well be "a model is worth a thousand datasets." Scientific models, such as Newtonian physics or biological gene regulatory networks, are human-driven simplifications of complex phenomena that serve as surrogates for the countless experiments that validated the models. Recently, machine learning has been able to overcome the inaccuracies of approximate modeling by directly learning the entire set of nonlinear interactions from data. However, without any predetermined structure from the scientific basis behind the problem, machine learning approaches are flexible but data-expensive, requiring large databases of homogeneous labeled training data. A central challenge is reconciling data that is at odds with simplified models without requiring "big data". In this work we develop a new methodology, universal differential equations (UDEs), which augments scientific models with machine-learnable structures for scientifically-based learning. We show how UDEs can be utilized to discover previously unknown governing equations, accurately extrapolate beyond the original data, and accelerate model simulation, all in a time and data-efficient manner. This advance is coupled with open-source software that allows for training UDEs which incorporate physical constraints, delayed interactions, implicitly-defined events, and intrinsic stochasticity in the model. Our examples show how a diverse set of computationally-difficult modeling issues across scientific disciplines, from automatically discovering biological mechanisms to accelerating climate simulations by 15,000x, can be handled by training UDEs.


In general, the policy iteration-based algorithms converge faster than value iteration-based algorithms #REFR , which is another reason for the superior performance of our approach. Our experimental results further support our arguments.

NNDP attacker's policy #OTHEREFR trains the model against one defensive plan at a time, due to which it forgets the previous plan. This way, it keeps learning and forgetting the plans.

However, we train our RL based attacker's policy against multiple defensive plans at a time, due to which it learns shared experience and performs better.

For RL agent, diverse environment configurations are only different in the "opening games", whereas the "end games" or "mid games" are likely to be similar across different environments.

The similarity in later stages can be utilized in parallel training, where the agent is trained against multiple environments simultaneously and gains shared experience, leading to faster convergence and improved performance.

Besides, NNDP approach is value iteration-based RL algorithm, whereas our approach is policy iteration-based RL algorithm.

Evolving Reinforcement Learning Environment To Minimize Learner'S Achievable Reward: An Application On Hardening Active Directory Systems

Abstract: We study a Stackelberg game between one attacker and one defender in a configurable environment. The defender picks a specific environment configuration. The attacker observes the configuration and attacks via Reinforcement Learning (RL trained against the observed environment). The defender's goal is to find the environment with minimum achievable reward for the attacker. We apply Evolutionary Diversity Optimization (EDO) to generate diverse population of environments for training. Environments with clearly high rewards are killed off and replaced by new offsprings to avoid wasting training time. Diversity not only improves training quality but also fits well with our RL scenario: RL agents tend to improve gradually, so a slightly worse environment earlier on may become better later. We demonstrate the effectiveness of our approach by focusing on a specific application, Active Directory (AD). AD is the default security management system for Windows domain networks. AD environment describes an attack graph, where nodes represent computers/accounts/etc., and edges represent accesses. The attacker aims to find the best attack path to reach the highest-privilege node. The defender can change the graph by removing a limited number of edges (revoke accesses). Our approach generates better defensive plans than the existing approach and scales better.

Reinforcement Learning: A Survey

Abstract: This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word ``reinforcement.'' The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state. It concludes with a survey of some implemented systems and an assessment of the practical utility of current methods for reinforcement learning.


Comparison Of Algorithm Variants: Elimination Order

The use of a completely random ordering was suggested in #REFR which introduced randomized approximate Gaussian elimination.

In this section, we investigate the impact of our approximate greedy elimination ordering (see Section 5.1) compared with a randomized ordering.

In #OTHEREFR , the random ordering plays a role in the proof that the algorithm produces a good factorization.

This paper also showed that one can randomize among the vertices with at most twice the average unweighted degree and still get a provably correct algorithm.

In our AC and other solvers, we instead eliminate a vertex with approximately minimum unweighted degree in each round.

To understand the impact of this choice, we compare AC with a variant that uses uniformly random elimination order, which we call AC-random.

In our first experiment, we reuse the setup from Section 5.4, i.e.

Robust And Practical Solution Of Laplacian Equations By Approximate Elimination *

Abstract: We introduce a new algorithm and software for solving linear equations in symmetric diagonally dominant matrices with non-positive off-diagonal entries (SDDM matrices), including Laplacian matrices. We use preconditioned conjugate gradient to solve the system of linear equations. Our preconditioner is a variant of the Approximate Cholesky factorization of Kyng and Sachdeva (FOCS 2016). Our factorization approach is simple: we eliminate matrix rows/columns one at a time, and update the remaining entries of the matrix by sampling entries to approximate the outcome of complete Cholesky factorization. Unlike earlier approaches, our sampled entries always maintain a connected graph on the neighbors of the eliminated variable. Our algorithm comes with a tuning parameter that upper bounds the number of samples made per original entry. We implement our solver algorithm in Julia, and experimentally evaluate its performance when using 1 or 2 samples for each original entry: We refer to these variants as AC and AC2 respectively. We investigate the performance of these implementations and compare their single-threaded performance to that of current stateof-the-art solvers including Combinatorial Multigrid (CMG), BoomerAMG-preconditioned Krylov solvers from HyPre and PETSc, Lean Algebraic Multigrid (LAMG), and MATLAB's preconditioned conjugate gradient with Incomplete Cholesky Factorization (ICC). Our experiments suggest that AC2 and AC attain a level of robustness and reliability not seen before in solvers for SDDM linear equations, while retaining good performance across all instances. Many large-scale evaluations of SDDM linear equation solvers have focused on solving problems on discretized 3D grids. We conduct evaluations across a much broader class of problems, including all large SDDM matrices from the SuiteSparse collection, as well as a broad array of programmatically generated instances. Our tests range up to 200 million non-zeros per system of linear equations. Our experiments show that AC and AC2 obtain good practical performance across many different types of SDDM-matrices, and significantly greater reliability than existing solvers. AC2 is the only solver that succeeds across all our tests, and it uses less than 7.2µs time per non-zero across to converge to 10 −8 relative residual error across all our experiments. AC is typically 1.5-2 times faster, but fails on one family of problems engineered to attack this algorithm. In our experiments using general sparse non-zero patterns, CMG, HyPre, PETSc, and ICC all fail on some instances from a majority of the families tested. Across these families, the median running time across different instances of AC2 and AC is comparable to or faster than other solvers. We also test the performance of our solvers on a wide array of Poisson problems on 3D grids, including grids with uniform, high-contrast, or anisotropic coefficients, and grids from the SPE Benchmark. Here, the CMG and HyPre solvers perform best, but AC and AC2 achieve worst case and median running times within a factor 4.1 and 6.2 of these respectively. Our code is public, and we detail precisely the set of tests we run and provide a tutorial on how to replicate the tests. We hope that others will adopt this suite of tests as a benchmark, which we refer to as SDDM2023.

Approximate Gaussian Elimination For Laplacians - Fast, Sparse, And Simple

Abstract: We show how to perform sparse approximate Gaussian elimination for Laplacian matrices. We present a simple, nearly linear time algorithm that approximates a Laplacian by a matrix with a sparse Cholesky factorization, the version of Gaussian elimination for symmetric matrices. This is the first nearly linear time solver for Laplacian systems that is based purely on random sampling, and does not use any graph theoretic constructions such as low-stretch trees, sparsifiers, or expanders. The crux of our analysis is a novel concentration bound for matrix martingales where the differences are sums of conditionally independent variables.


While simulators allow recording synchronized RGB-D and IMU data #REFR , manually gathering sufficient training data is difficult.

A key challenge in acquiring training data for the SML network is the lack of RGB-D+IMU datasets.

In our pipeline, IMU data is needed to run VIO to generate sparse metric depth.

We select TartanAir #OTHEREFR for its large size and variety of outdoor and indoor sequences. IMU data is not provided in this dataset.

To proxy sparse depth map generation, we run the VINS-Mono feature tracker front-end #OTHEREFR to obtain sparse feature locations and then sample ground truth depth at those locations.

We use a 70%-30% train-test split for TartanAir, with 172K training and 73K test samples taken from both easy and hard sequences.

In addition to the synthetic TartanAir dataset, we benchmark on VOID #OTHEREFR , which offers real-world data collected using an Intel RealSense D435i camera and the VIO system XIVO #OTHEREFR .

This dataset is smaller than TartanAir, with only 47K training and 800 test samples. We use the published train-test split.

Monocular Visual-Inertial Depth Estimation

Abstract: We present a visual-inertial depth estimation pipeline that integrates monocular depth estimation and visualinertial odometry to produce dense depth estimates with metric scale. Our approach performs global scale and shift alignment against sparse metric depth, followed by learning-based dense alignment. We evaluate on the TartanAir and VOID datasets, observing up to 30% reduction in inverse RMSE with dense scale alignment relative to performing just global alignment alone. Our approach is especially competitive at low density; with just 150 sparse metric depth points, our dense-to-dense depth alignment method achieves over 50% lower iRMSE over sparse-to-dense depth completion by KBNet, currently the state of the art on VOID. We demonstrate successful zero-shot transfer from synthetic TartanAir to real-world VOID data and perform generalization tests on NYUv2 and VCU-RVI. Our approach is modular and is compatible with a variety of monocular depth estimation models.

Airsim: High-Fidelity Visual And Physical Simulation For Autonomous Vehicles

Abstract: Developing and testing algorithms for autonomous vehicles in real world is an expensive and time consuming process. Also, in order to utilize recent advances in machine intelligence and deep learning we need to collect a large amount of annotated training data in a variety of conditions and environments. We present a new simulator built on Unreal Engine that offers physically and visually realistic simulations for both of these goals. Our simulator includes a physics engine that can operate at a high frequency for real-time hardware-in-the-loop (HITL) simulations with support for popular protocols (e.g. MavLink). The simulator is designed from the ground up to be extensible to accommodate new types of vehicles, hardware platforms and software protocols. In addition, the modular design enables various components to be easily usable independently in other projects. We demonstrate the simulator by first implementing a quadrotor as an autonomous vehicle and then experimentally comparing the software components with real-world flights.


The Siamese network consists of an encoder network f and a prediction MLP h, where the encoder includes a backbone model (e.g., ResNet #REFR ) and a projection MLP.

Self-supervised learning aims to learn visual representation without data labeling cost.

Recent advances #OTHEREFR show that self-supervised learning can achieve similar or even better performance than supervised representation learning.

A common strategy of these methods is to learn representations that are invariant under different data augmentations by maximizing their similarity with contrastive loss optimization. However, these approaches require largesized batches and negative samples.

SimSiam #OTHEREFR addresses this issue by utilizing the stop-gradient technique to prevent the collapsing of Siamese networks.

Given two randomly augmented views of x 1 and x 2 from an input image x, Simsiam aims to minimize the negative cosine similarity between the predictor output p 1 (p 1 = f (h(x 1 ))) and the projector output z 2 (z 2 = f (x 2 )) with a symmetrized loss as:

LSSL = 1 2 D(p1, stopgrad(z2)) + 1 2 D(p2, stopgrad(z1)) (1)

where D is a negative cosine similarity function.

Given the distorted versions of an instance, BarlowTwin #OTHEREFR minimizes the redundancy between their embedding vector components while conserving the maximum information.

This can be achieved by making the cross-correlation matrix, computed between the outputs of two identical networks, closer to the identity matrix, through the minimization of the following loss:

Efficient Self-Supervised Continual Learning With Progressive Task-Correlated Layer Freezing

Abstract: Inspired by the success of Self-supervised learning (SSL) in learning visual representations from unlabeled data, a few recent works have studied SSL in the context of continual learning (CL), where multiple tasks are learned sequentially, giving rise to a new paradigm, namely selfsupervised continual learning (SSCL). It has been shown that the SSCL outperforms supervised continual learning (SCL) as the learned representations are more informative and robust to catastrophic forgetting. However, if not designed intelligently, the training complexity of SSCL may be prohibitively high due to the inherent training cost of SSL. In this work, by investigating the task correlations in SSCL setup first, we discover an interesting phenomenon that, with the SSL-learned background model, the intermediate features are highly correlated between tasks. Based on this new finding, we propose a new SSCL method with layer-wise freezing which progressively freezes partial layers with the highest correlation ratios for each task to improve training computation efficiency and memory efficiency. Extensive experiments across multiple datasets are performed, where our proposed method shows superior performance against the SoTA SSCL methods under various SSL frameworks. For example, compared to LUMP, our method achieves 12%/14%/12% GPU training time reduction, 23%/26%/24% memory reduction, 35%/34%/33% backward FLOPs reduction, and 1.31%/1.98%/1.21% forgetting reduction without accuracy degradation on three datasets, respectively.

Deep Residual Learning For Image Recognition

Abstract: Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.


We compare 2 , 1 and LPIPS #REFR distances over CIFAR10 dataset, and presente the results in Tab. 8.

In TETRA's second phase, we calculate the distance between the input image and the transformed images, and we classify based on the shortest one.

Hence, the distance metric that we use for the classification is important.

Different metrics have different properties, and we aim at a distance metric that is able to measure the semantic distance between images.

We compare the results using the following defense methods #OTHEREFR .

As demonstrated, 2 distance metric performs better, therefore is a favorable choice. Table 8 . CIFAR10 results. In the first column, we state the method.

For every base method, we report three consecutive lines of results.

One for the base method and then two TETRA distance metric variations used for classification: L2 and LPIPS #OTHEREFR .

In the next columns, we state the architecture, the trained threat model (TTM), and four attacks with different threat models.

Abstract: It has been recently discovered that adversarially trained classifiers exhibit an intriguing property, referred to as perceptually aligned gradients (PAG). PAG implies that the gradients of such classifiers possess a meaningful structure, aligned with human perception. Adversarial training is currently the best-known way to achieve classification robustness under adversarial attacks. The PAG property, however, has yet to be leveraged for further improving classifier robustness. In this work, we introduce Classifier Robustness Enhancement Via Test-Time Transformation (TETRA)-a novel defense method that utilizes PAG, enhancing the performance of trained robust classifiers. Our method operates in two phases. First, it modifies the input image via a designated targeted adversarial attack into each of the dataset's classes. Then, it classifies the input image based on the distance to each of the modified instances, with the assumption that the shortest distance relates to the true class. We show that the proposed method achieves state-of-the-art results and validate our claim through extensive experiments on a variety of defense methods, classifier architectures, and datasets. We also empirically demonstrate that TETRA can boost the accuracy of any differentiable adversarial training classifier across a variety of attacks, including ones unseen at training. Specifically, applying TETRA leads to substantial improvement of up to +23%, +20%, and +26% on CIFAR10, CIFAR100, and ImageNet, respectively.

The Unreasonable Effectiveness Of Deep Features As A Perceptual Metric

Abstract: While it is nearly effortless for humans to quickly assess the perceptual similarity between two images, the underlying processes are thought to be quite complex. Despite this, the most widely used perceptual metrics today, such as PSNR and SSIM, are simple, shallow functions, and fail to account for many nuances of human perception. Recently, the deep learning community has found that features of the VGG network trained on the ImageNet classification task has been remarkably useful as a training loss for image synthesis. But how perceptual are these so-called"perceptual losses"? What elements are critical for their success? To answer these questions, we introduce a new Full Reference Image Quality Assessment (FR-IQA) dataset of perceptual human judgments, orders of magnitude larger than previous datasets. We systematically evaluate deep features across different architectures and tasks and compare them with classic metrics. We find that deep features outperform all previous metrics by huge margins. More surprisingly, this result is not restricted to ImageNet-trained VGG features, but holds across different deep architectures and levels of supervision (supervised, self-supervised, or even unsupervised). Our results suggest that perceptual similarity is an emergent property shared across deep visual representations.


On the other hand, it requires the more challenging Dec-POMDP formalization of standard MARL #REFR when letting several agents act simultaneously.

Let S and A represent the global state and action spaces, respectively, defined as the joint and union of the respective agents' local spaces, S = ∈V S and A = ∈V A .

The theoretical framework of MAGNNETO allows to implement both Q-learning and PG methods, so for the sake of generalization let represent the global RL-based function that is aimed to learn -i.e., the global state-action value function for the former, or the global policy for the latter.

A main contribution of MAGNNETO is that it makes all agents ∈ V learn the global RL-based function approximator in a fully distributed fashion -i.e., all agents end up constructing and having access to the very same representation .

In particular, and from a theoretical RL standpoint, this allows to formulate the problem within two different paradigms depending on the number of actions allowed at each time-step of the RL episode.

On the one hand, imposing a single action per time-step enables to devise the problem as a time-homogeneous MDP of single-agent RL #OTHEREFR .

Note, however, that in practice the execution pipeline of MAGNNETO is exactly the same in both cases.

Another relevant feature of our design is that all agents ∈ V are able to internally construct such global representation mainly through message communications with their direct neighboring agents B ( ) and their local computations, no longer needing a centralized entity responsible for collecting and processing all the global information together.

Such a decentralized, message-based generation of the global function is achieved by modeling the global function with a MPNN (see Sec.

III-A3), so that all agents ∈ V deployed in the network are actually replicas of the MPNN modules (message, aggregation, update and readout functions) that perform regular message exchanges with their neighbors B ( ) following the message passing iteration procedure of MPNNs; in particular, note that such parameter sharing implies that all agents share as well the same local state and action spaces.

This reinterpretation of a MPNN as a set of copies of its internal modules is especially important due to the fact that in our approach we directly map the graph G to a real networked scenario, deploying copies of the MPNN modules along hardware devices in the network (e.g., routers) and making all message communications involved to actually go through the real network infrastructure.

Magnneto: A Graph Neural Network-Based Multi-Agent System For Traffic Engineering

Abstract: Current trends in networking propose the use of Machine Learning (ML) for a wide variety of network optimization tasks. As such, many efforts have been made to produce ML-based solutions for Traffic Engineering (TE), which is a fundamental problem in ISP networks. Nowadays, state-of-the-art TE optimizers rely on traditional optimization techniques, such as Local search, Constraint Programming, or Linear programming. In this paper, we present MAGNNETO, a distributed ML-based framework that leverages Multi-Agent Reinforcement Learning and Graph Neural Networks for distributed TE optimization. MAGNNETO deploys a set of agents across the network that learn and communicate in a distributed fashion via message exchanges between neighboring agents. Particularly, we apply this framework to optimize link weights in OSPF, with the goal of minimizing network congestion. In our evaluation, we compare MAGNNETO against several state-of-the-art TE optimizers in more than 75 topologies (up to 153 nodes and 354 links), including realistic traffic loads. Our experimental results show that, thanks to its distributed nature, MAGNNETO achieves comparable performance to state-of-the-art TE optimizers with significantly lower execution times. Moreover, our ML-based solution demonstrates a strong generalization capability to successfully operate in new networks unseen during training.

Learning To Communicate With Deep Multi-Agent Reinforcement Learning

Abstract: We consider the problem of multiple agents sensing and acting in environments with the goal of maximising their shared utility. In these environments, agents must learn communication protocols in order to share information that is needed to solve the tasks. By embracing deep neural networks, we are able to demonstrate end-to-end learning of protocols in complex environments inspired by communication riddles and multi-agent computer vision problems with partial observability. We propose two approaches for learning in these domains: Reinforced Inter-Agent Learning (RIAL) and Differentiable Inter-Agent Learning (DIAL). The former uses deep Q-learning, while the latter exploits the fact that, during learning, agents can backpropagate error derivatives through (noisy) communication channels. Hence, this approach uses centralised learning but decentralised execution. Our experiments introduce new environments for studying the learning of communication protocols and present a set of engineering innovations that are essential for success in these domains.

challenging Dec-POMDP formalization
several agents


This reinforces the findings that rotation forest is the most effective classifier for problems with continuous features #REFR .

A hierarchical window is run over the two augmented series, with the signature transform being applied to each window.

The output for each window is then concatenated into a feature vector.

Figure 6 shows the relative rank performance, and Table 3 summarises the overall performance statistics.

All four pipelines are significantly more accurate than 1-NN DTW.

Excluding feature extraction and using rotation forest rather than random forest with TSFresh increases accuracy by over 0.05.

An example of a problem where interval based approaches may be superior.

Each series is a spectrogram from a bottle of alcohol with a different concentration of ethanol.

The discriminatory features are in the near infrared interval (green box to the right).

However, the confounding factors such as bottle shape, labelling and colouring cause variation in the visible range (red box to the left).

Using intervals containing just the near infrared features is likely to make classification easier.

Bake Off Redux: A Review And Experimental Evaluation Of Recent Time Series Classification Algorithms

Abstract: In 2017, a research paper (Bagnall et al., 2017) compared 18 Time Series Classification (TSC) algorithms on 85 datasets from the University of California, Riverside (UCR) archive. This study, commonly referred to as a 'bake off', identified that only nine algorithms performed significantly better than the Dynamic Time Warping (DTW) and Rotation Forest benchmarks that were used. The study categorised each algorithm by the type of feature they extract from time series data, forming a taxonomy of five main algorithm types. This categorisation of algorithms alongside the provision of code and accessible results for reproducibility has helped fuel an increase in popularity of the TSC field. Over six years have passed since this bake off, the UCR archive has expanded to 112 datasets and there have been a large number of new algorithms proposed. We revisit the bake off, seeing how each of the proposed categories have advanced since the original publication, and evaluate the performance of newer algorithms against the previous best-of-category using an expanded UCR archive. We extend the taxonomy to include three new categories to reflect recent developments. Alongside the originally proposed distance, interval, shapelet, dictionary and hybrid based algorithms, we compare newer convolution and feature based algorithms as well as deep learning approaches. We introduce 30 classification datasets either recently donated to the archive or reformatted to the TSC format, and use these to further evaluate the best performing algorithm from each category. Overall, we find that two recently proposed algorithms, Hydra+MultiROCKET Dempster et al. (2022) and HIVE-COTEv2 Middlehurst et al. (2021), perform significantly better than other approaches on both the current and new TSC problems.

Is Rotation Forest The Best Classifier For Problems With Continuous Features?

Abstract: Rotation forest is a tree based ensemble that performs transforms on subsets of attributes prior to constructing each tree. We present an empirical comparison of classifiers for problems with only real valued features. We evaluate classifiers from three families of algorithms: support vector machines; tree-based ensembles; and neural networks. We compare classifiers on unseen data based on the quality of the decision rule (using classification error) the ability to rank cases (area under the receiver operator curve) and the probability estimates (using negative log likelihood). We conclude that, in answer to the question posed in the title, yes, rotation forest, is significantly more accurate on average than competing techniques when compared on three distinct sets of datasets. The same pattern of results are observed when tuning classifiers on the train data using a grid search. We investigate why rotation forest does so well by testing whether the characteristics of the data can be used to differentiate classifier performance. We assess the impact of the design features of rotation forest through an ablative study that transforms random forest into rotation forest. We identify the major limitation of rotation forest as its scalability, particularly in number of attributes. To overcome this problem we develop a model to predict the train time of the algorithm and hence propose a contract version of rotation forest where a run time cap {\em a priori}. We demonstrate that on large problems rotation forest can be made an order of magnitude faster without significant loss of accuracy and that there is no real benefit (on average) from tuning the ensemble. We conclude that without any domain knowledge to indicate an algorithm preference, rotation forest should be the default algorithm of choice for problems with continuous attributes.


LiDAR 3D and 4D Panoptic segmentation 3D Panoptic segmentation The task of panoptic segmentation was first proposed in the image domain, and was later extended to LiDAR point clouds after a large-scale outdoor LiDAR point cloud dataset, SemanticKITTI, was published with panoptic labels #REFR .

Similar to the semantic segmentation #OTHEREFR and panoptic segmentation techniques in the image domain #OTHEREFR , their 3D counterparts can be classified into proposal-based and proposal-free methods.

Proposal-based methods #OTHEREFR require an object detection module to locate the objects first and then predict the instance mask for each bounding box and conduct semantic segmentation on the background pixels.

This strategy needs to deal with potential conflicts among the segmentations.

More methods fall into the other category, proposalfree methods, which conduct semantic segmentation first and then cluster the points belonging to different instances.

A lot of research efforts focused on clustering strategies, as it impacts overall efficiency and performance.

4D Panoptic Segmentation As Invariant And Equivariant Field Prediction

Abstract: In this paper, we develop rotation-equivariant neural networks for 4D panoptic segmentation. 4D panoptic segmentation is a recently established benchmark task for autonomous driving, which requires recognizing semantic classes and object instances on the road based on LiDAR scans, as well as assigning temporally consistent IDs to instances across time. We observe that the driving scenario is symmetric to rotations on the ground plane. Therefore, rotation-equivariance could provide better generalization and more robust feature learning. Specifically, we review the object instance clustering strategies, and restate the centerness-based approach and the offset-based approach as the prediction of invariant scalar fields and equivariant vector fields. Other sub-tasks are also unified from this perspective, and different invariant and equivariant layers are designed to facilitate their predictions. Through evaluation on the standard 4D panoptic segmentation benchmark of SemanticKITTI, we show that our equivariant models achieve higher accuracy with lower computational costs compared to their non-equivariant counterparts. Moreover, our method sets the new state-of-the-art performance and achieves 1st place on the SemanticKITTI 4D Panoptic Segmentation leaderboard.

A Benchmark For Lidar-Based Panoptic Segmentation Based On Kitti

Abstract: Panoptic segmentation is the recently introduced task that tackles semantic segmentation and instance segmentation jointly. In this paper, we present an extension of SemanticKITTI, which is a large-scale dataset providing dense point-wise semantic labels for all sequences of the KITTI Odometry Benchmark, for training and evaluation of laser-based panoptic segmentation. We provide the data and discuss the processing steps needed to enrich a given semantic annotation with temporally consistent instance information, i.e., instance information that supplements the semantic labels and identifies the same instance over sequences of LiDAR point clouds. Additionally, we present two strong baselines that combine state-of-the-art LiDAR-based semantic segmentation approaches with a state-of-the-art detector enriching the segmentation with instance information and that allow other researchers to compare their approaches against. We hope that our extension of SemanticKITTI with strong baselines enables the creation of novel algorithms for LiDAR-based panoptic segmentation as much as it has for the original semantic segmentation and semantic scene completion tasks. Data, code, and an online evaluation using a hidden test set will be published on http://semantic-kitti.org.


Afterwards, they transform the functional scenarios into logical scenarios 2 by adding actions and events, specifying parameter ranges, as well as object and parameter dependencies #REFR .

Their ontology covers the road infrastructure including slope, surface and lane, as well as the ego vehicle with its position and speed.

#OTHEREFR construct an ontology which models the five layers of a scenario.

These are the road, the traffic infrastructure, the temporary manipulation of the road and traffic infrastructure for example through construction sites, the static and dynamic objects and the environment layer.

They define the degree of complexity, the amount of positions per lane, the amount of traffic participants and the abstraction level before they use combinatorial techniques and permutations to generate a start and an end scene of a functional scenario #OTHEREFR .

Chen and Kloul #OTHEREFR use a three layered methodology for the generation of test scenarios.

The first layer consists of a highway, a weather and a vehicle ontology to model the static and mobile concepts of a scene.

The second layer describes the interactions between the static and mobile concepts by the use of rules expressed in first-order-logic.

The third layer is the generation layer, which adds several scenes to a scenario by considering actions and events.

The simulation scenario generation framework of Medrano-Berumen and Akbas #OTHEREFR uses a matrix where each row represents a road piece or actor in a semantic string.

A Comprehensive Review On Ontologies For Scenario-Based Testing In The Context Of Autonomous Driving

Abstract: The verification and validation of autonomous driving vehicles remains a major challenge due to the high complexity of autonomous driving functions. Scenario-based testing is a promising method for validating such a complex system. Ontologies can be utilized to produce test scenarios that are both meaningful and relevant. One crucial aspect of this process is selecting the appropriate method for describing the entities involved. The level of detail and specific entity classes required will vary depending on the system being tested. It is important to choose an ontology that properly reflects these needs. This paper summarizes key representative ontologies for scenario-based testing and related use cases in the field of autonomous driving. The considered ontologies are classified according to their level of detail for both static facts and dynamic aspects. Furthermore, the ontologies are evaluated based on the presence of important entity classes and the relations between them.

From Functional To Logical Scenarios: Detailing A Keyword-Based Scenario Description For Execution In A Simulation Environment

Abstract: Scenario-based development and test processes are a promising approach for verifying and validating automated driving functions. For this purpose, scenarios have to be generated during the development process in a traceable manner. In early development stages, the operating scenarios of the item to be developed are usually described in an abstract, linguistic way.Within the scope of a simulation-assisted test process, these linguistically described scenarios have to be transformed into a state space representation and converted into data formats which can be used with the respective simulation environment. Currently, this step of detailing scenarios takes a considerable manual effort. Furthermore, a standardized interpretation of the linguistically described scenarios and a consistent transformation into the data formats are not guaranteed due to multiple authors as well as many constraints between the scenario parameters. In this paper, the authors present an approach to automatically detail a keyword-based scenario description for execution in a simulation environment and provide a basis for test case generation. As a first step, the keyword-based description is transformed into a parameter space representation. At the same time, constraints regarding the selection and combination of parameter values are documented for the following process steps (e. g. evolutionary or stochastic test methods). As a second step, the parameter space representation is converted into data formats required by the simulation environment. As an example, the authors use scenarios on German freeways and convert them into the data formats OpenDRIVE (description of the road) and OpenSCENARIO (description of traffic participants and environmental conditions) for execution in the simulation environment Virtual Test Drive.


This scenario has two main advantages, (i) the loss is computed in the representation space, which is in general lower dimensional and avoids focusing on small details of the input and (ii) views can be defined to capture different aspects of the data #REFR .

The InfoMax principle is extended to a multi-view approach, in which rather than maximizing the MI between the input and output of the network, the agreement is maximized between the representations of two different views of the input.

Deep InfoMax #OTHEREFR trains an encoder maximizing the average mutual information between local patches and global representations of an image.

Deep Graph InfoMax #OTHEREFR and InfoGraph #OTHEREFR extend this work to the graph domain, targetting the MI between node and graph level embeddings.

The graph representation is obtained with a global pooling layer applied to the local node embeddings.

In DGI, since most datasets consist of one single graph, the authors create a corrupted version of the graph by shuffling the node features and contrasting negative and positive pairs.

Rgi : Regularized Graph Infomax For Self-Supervised Learning On Graphs

Abstract: Self-supervised learning is gaining considerable attention as a solution to avoid the requirement of extensive annotations in representation learning on graphs. We introduce Regularized Graph Infomax (RGI), a simple yet effective framework for node level self-supervised learning on graphs that trains a graph neural network encoder by maximizing the mutual information between node level local and global views, in contrast to previous works that employ graph level global views. The method promotes the predictability between views while regularizing the covariance matrices of the representations. Therefore, RGI is non-contrastive, does not depend on complex asymmetric architectures nor training tricks, is augmentation-free and does not rely on a two branch architecture. We run RGI on both transductive and inductive settings with popular graph benchmarks and show that it can achieve state-of-the-art performance regardless of its simplicity.

On Mutual Information Maximization For Representation Learning

Abstract: Many recent methods for unsupervised or self-supervised representation learning train feature extractors by maximizing an estimate of the mutual information (MI) between different views of the data. This comes with several immediate problems: For example, MI is notoriously hard to estimate, and using it as an objective for representation learning may lead to highly entangled representations due to its invariance under arbitrary invertible transformations. Nevertheless, these methods have been repeatedly shown to excel in practice. In this paper we argue, and provide empirical evidence, that the success of these methods cannot be attributed to the properties of MI alone, and that they strongly depend on the inductive bias in both the choice of feature extractor architectures and the parametrization of the employed MI estimators. Finally, we establish a connection to deep metric learning and argue that this interpretation may be a plausible explanation for the success of the recently introduced methods.


However, their results show it is not able to reach the same level of performance as #REFR .

The conventional perceptron updates its weights for every prediction, which can over-weight the final example.

The averaged perception keeps a running weighted sum of the obtained feature weights through all training examples and iterations.

The final weights are obtained by dividing the weighted sum by the number of iterations. #OTHEREFR

(2019) propose another NER model based on a long short-term memory (LSTM) architecture.

The authors argue that it can be more useful when the data set has more variation, as the LSTM model does not require time-consuming feature engineering.

SecBERT 4 is a pre-trained encoder trained on a large corpus of cybersecurity texts.

It is based on the BERT architecture #OTHEREFR and uses a vocabulary specialized for cybersecurity.

SecBERT can be fine-tuned for specific tasks such as NER.

Another pre-trained encoder similar to SecBERT is SecureBERT, proposed by #OTHEREFR .

SecureBERT leverages a customized tokenizer and an approach to alter pre-trained weights.

Constructing A Knowledge Graph From Textual Descriptions Of Software Vulnerabilities In The National Vulnerability Database

Abstract: Knowledge graphs have shown promise for several cybersecurity tasks, such as vulnerability assessment and threat analysis. In this work, we present a new method for constructing a vulnerability knowledge graph from information in the National Vulnerability Database (NVD). Our approach combines named entity recognition (NER), relation extraction (RE), and entity prediction using a combination of neural models, heuristic rules, and knowledge graph embeddings. We demonstrate how our method helps to fix missing entities in knowledge graphs used for cybersecurity and evaluate the performance.

Automatic Labeling For Entity Extraction In Cyber Security

Abstract: Timely analysis of cyber-security information necessitates automated information extraction from unstructured text. While state-of-the-art extraction methods produce extremely accurate results, they require ample training data, which is generally unavailable for specialized applications, such as detecting security related entities; moreover, manual annotation of corpora is very costly and often not a viable solution. In response, we develop a very precise method to automatically label text from several data sources by leveraging related, domainspecific, structured data and provide public access to a corpus annotated with cyber-security entities. Next, we implement a Maximum Entropy Model trained with the average perceptron on a portion of our corpus ( 750,000 words) and achieve near perfect precision, recall, and accuracy, with training times under 17 seconds.


Self-Supervised Learning Vs Supervised Learning On Limited Labeled Data

The above findings are consistent with those reported in #REFR that the advantage of the self-supervised learning-based method is its better regularisation on low data regimes to avoid overfitting problems compared to fully-supervised methods.

The resulting average accuracy and the corresponding standard deviation of all compared models are illustrated in Fig. 7 .

First, our finetuned model consistently outperforms other supervised learning-based models for sample sizes varying from 1 to 1000 on the emotion recognition tasks of all three datasets.

Among supervised learning-based methods, SimpDCNN exhibited the poorest results, over which our SSL model could achieve significant performance gains of 6.84% -21.19% for different downstream tasks.

Our fully-supervised model yields the highest results compared to other supervised models, whereas the fine-tuned model initialized by self-supervised learning parameters continues to enhance performance by 5.24% -13.63%.

Second, for all downstream tasks, the standard deviation obtained by our fine-tuned model is narrower with respect to the supervised learningbased deep models, demonstrating its superior generalization ability across different samples.

As the amount of available labeled data increases, the difference in performance between the two types of models gradually decreases.

Overall, the comparison results suggest that the proposed method can produce more meaningful and robust representations for wearable emotion recognition than fullysupervised methods, offering a potential solution to the problem of little labeled data.

Transformer-Based Self-Supervised Multimodal Representation Learning For Wearable Emotion Recognition

Abstract: Recently, wearable emotion recognition based on peripheral physiological signals has drawn massive attention due to its less invasive nature and its applicability in real-life scenarios. However, how to effectively fuse multimodal data remains a challenging problem. Moreover, traditional fully-supervised based approaches suffer from overfitting given limited labeled data. To address the above issues, we propose a novel self-supervised learning (SSL) framework for wearable emotion recognition, where efficient multimodal fusion is realized with temporal convolution-based modality-specific encoders and a transformer-based shared encoder, capturing both intra-modal and intermodal correlations. Extensive unlabeled data is automatically assigned labels by five signal transforms, and the proposed SSL model is pre-trained with signal transformation recognition as a pretext task, allowing the extraction of generalized multimodal representations for emotion-related downstream tasks. For evaluation, the proposed SSL model was first pre-trained on a large-scale self-collected physiological dataset and the resulting encoder was subsequently frozen or fine-tuned on three public supervised emotion recognition datasets. Ultimately, our SSL-based method achieved state-of-the-art results in various emotion classification tasks. Meanwhile, the proposed model was proved to be more accurate and robust compared to fully-supervised methods on low data regimes.

How Useful Is Self-Supervised Pretraining For Visual Tasks?

Abstract: Recent advances have spurred incredible progress in self-supervised pretraining for vision. We investigate what factors may play a role in the utility of these pretraining methods for practitioners. To do this, we evaluate various self-supervised algorithms across a comprehensive array of synthetic datasets and downstream tasks. We prepare a suite of synthetic data that enables an endless supply of annotated images as well as full control over dataset difficulty. Our experiments offer insights into how the utility of self-supervision changes as the number of available labels grows as well as how the utility changes as a function of the downstream task and the properties of the training data. We also find that linear evaluation does not correlate with finetuning performance. Code and data is available at \href{https://www.github.com/princeton-vl/selfstudy}{github.com/princeton-vl/selfstudy}.

self-supervised learning-based method
fully-supervised methods


Results On Small Amount Of Fine-Tuning Data

Both T-Dec #REFR and Tac, which do not have the opportunity to pre-learn a sufficient capability of attention alignment in pre-training, show similarly lower performance than the others.

Table 1 presents the superior performance of the proposed methods compared to competing methods on small amounts of fine-tuning data.

Without data augmentation during finetuning, T-SD outperforms all unsupervised pre-training methods and Tac.

In contrast, the proposed de-warping task encourages the model to learn both preliminary knowledge of attention alignment and autoregressive prediction.

When data augmentation is applied during fine-tuning, T-SD with SegAug outperforms other combinations of pre-training and augmentation methods.

SegAug even effectively improves the performance of other pre-training baselines and shows competitive performance compared to other augmentation methods.

Table 2 shows the preference test results with competitive methods using 0.5 shards of fine-tuning data.

Unsupervised Pre-Training For Data-Efficient Text-To-Speech On Low Resource Languages

Abstract: Neural text-to-speech (TTS) models can synthesize natural human speech when trained on large amounts of transcribed speech. However, collecting such large-scale transcribed data is expensive. This paper proposes an unsupervised pre-training method for a sequenceto-sequence TTS model by leveraging large untranscribed speech data. With our pre-training, we can remarkably reduce the amount of paired transcribed data required to train the model for the target downstream TTS task. The main idea is to pre-train the model to reconstruct de-warped mel-spectrograms from warped ones, which may allow the model to learn proper temporal assignment relation between input and output sequences. In addition, we propose a data augmentation method that further improves the data efficiency in finetuning. We empirically demonstrate the effectiveness of our proposed method in low-resource language scenarios, achieving outstanding performance compared to competing methods. The code and audio samples are

Semi-Supervised Training For Improving Data Efficiency In End-To-End Speech Synthesis

Abstract: Although end-to-end text-to-speech (TTS) models such as Tacotron have shown excellent results, they typically require a sizable set of high-quality pairs for training, which are expensive to collect. In this paper, we propose a semi-supervised training framework to improve the data efficiency of Tacotron. The idea is to allow Tacotron to utilize textual and acoustic knowledge contained in large, publicly-available text and speech corpora. Importantly, these external data are unpaired and potentially noisy. Specifically, first we embed each word in the input text into word vectors and condition the Tacotron encoder on them. We then use an unpaired speech corpus to pre-train the Tacotron decoder in the acoustic domain. Finally, we fine-tune the model using available paired data. We demonstrate that the proposed framework enables Tacotron to generate intelligible speech using less than half an hour of paired training data.


In particular, SAC (EP) with seed optimization most closely matches the results reported by #REFR .

(2018) , we used seed optimization in Figure 15 : When searching seeds, we were able to achieve average performance and shaded regions much closer to that reported in the original SAC work.

Solid lines denote average performance, and shaded regions denote minimum and maximum performance.

As a note, this is bad practice; we only conduct seed search in the name of reproduction.

In Figure 15 , where we chose the best 5 seeds of 30 for each algorithm, the results more closely match those from the original paper.

We now turn to running an experiment that more closely resembles the principles laid out in this document, especially with respect to reporting performance of tuned baselines. Although we do not know how #OTHEREFR

(2018) tuned the baseline algorithms in their experiments, we found that the performance of DDPG on Half Cheetah was under-reported in this work.

In the experiments here, we use the tuned hyperparameters for DDPG as reported by SpinningUp baselines 22 .

Since Gaussian noise is known to outperform OU noise in some cases #OTHEREFR 23 , we use uncorrelated, unbounded Gaussian noise for action exploration in DDPG instead of OU noise.

Furthermore, we try both SAC and DDPG with an exploration phase at the beginning of the experiment, where an action is drawn uniformly randomly for the first 10,000 steps.

Empirical Design In Reinforcement Learning

Abstract: Empirical design in reinforcement learning is no small task. Running good experiments requires attention to detail and at times significant computational resources. While compute resources available per dollar have continued to grow rapidly, so have the scale of typical experiments in reinforcement learning. It is now common to benchmark agents with millions of parameters against dozens of tasks, each using the equivalent of 30 days of experience. The scale of these experiments often conflict with the need for proper statistical evidence, especially when comparing algorithms. Recent studies have highlighted how popular algorithms are sensitive to hyper-parameter settings and implementation details, and that common empirical practice leads to weak statistical evidence (Machado et al., 2018; Henderson et al., 2018). Here we take this one step further. This manuscript represents both a call to action, and a comprehensive resource for how to do good experiments in reinforcement learning. In particular, we cover: the statistical assumptions underlying common performance measures, how to properly characterize performance variation and stability, hypothesis testing, special considerations for comparing multiple agents, baseline and illustrative example construction, and how to deal with hyperparameters and experimenter bias. Throughout we highlight common mistakes found in the literature and the statistical consequences of those in example experiments. The objective of this document is to provide answers on how we can use our unprecedented compute to do good science in reinforcement learning, as well as stay alert to potential pitfalls in our empirical design.

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning With A Stochastic Actor

Abstract: Model-free deep reinforcement learning (RL) algorithms have been demonstrated on a range of challenging decision making and control tasks. However, these methods typically suffer from two major challenges: very high sample complexity and brittle convergence properties, which necessitate meticulous hyperparameter tuning. Both of these challenges severely limit the applicability of such methods to complex, real-world domains. In this paper, we propose soft actor-critic, an off-policy actor-critic deep RL algorithm based on the maximum entropy reinforcement learning framework. In this framework, the actor aims to maximize expected reward while also maximizing entropy - that is, succeed at the task while acting as randomly as possible. Prior deep RL methods based on this framework have been formulated as Q-learning methods. By combining off-policy updates with a stable stochastic actor-critic formulation, our method achieves state-of-the-art performance on a range of continuous control benchmark tasks, outperforming prior on-policy and off-policy methods. Furthermore, we demonstrate that, in contrast to other off-policy algorithms, our approach is very stable, achieving very similar performance across different random seeds.


It has been proven in #REFR that Grad-CAM and original CAM produce equivalent results in network with GAP layer.

In light of the challenge of annotating all groundtruth labels for an image, multi-label learning in the presence of missing labels (MLML) has also attracted much research attention #OTHEREFR .

Class Activation Mapping (CAM) is a technique to obtain discriminative regions for specific classes in an image and generate class activation maps (CAMs).

The original CAM #OTHEREFR operates a weighted sum on the feature maps extracted by the backbone network.

It is restricted to networks with global average pooling (GAP) layer.

Grad-CAM #OTHEREFR utilizes local gradient to generate CAMs in any architecture without the need for re-training.

As GAP layer is a common structure in classification model, original CAM is suitable for most scenarios.

Furthermore, there are also explorations on gradient-free extensions #OTHEREFR .

The CAM technique derived from classification network has been widely used for weakly supervised visual tasks, such as weakly supervised object location #OTHEREFR and object segmentation #OTHEREFR .

In both tasks, category labels are employed as supervision, and CAM-based localization serves as supplementary information.

Knowledge Distillation From Single To Multi Labels: An Empirical Study

Abstract: Knowledge distillation (KD) has been extensively studied in single-label image classification. However, its efficacy for multi-label classification remains relatively unexplored. In this study, we firstly investigate the effectiveness of classical KD techniques, including logit-based and feature-based methods, for multi-label classification. Our findings indicate that the logit-based method is not wellsuited for multi-label classification, as the teacher fails to provide inter-category similarity information or regularization effect on student model's training. Moreover, we observe that feature-based methods struggle to convey compact information of multiple labels simultaneously. Given these limitations, we propose that a suitable dark knowledge should incorporate class-wise information and be highly correlated with the final classification results. To address these issues, we introduce a novel distillation method based on Class Activation Maps (CAMs), which is both effective and straightforward to implement. Across a wide range of settings, CAMs-based distillation consistently outperforms other methods. Code is available at https://github.com/yzqinjacob/Distill-MLC.

Grad-Cam: Visual Explanations From Deep Networks Via Gradient-Based Localization

Abstract: We propose a technique for producing ‘visual explanations’ for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable. Our approach—Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept (say ‘dog’ in a classification network or a sequence of words in captioning network) flowing into the final convolutional layer to produce a coarse localization map highlighting the important regions in the image for predicting the concept. Unlike previous approaches, Grad-CAM is applicable to a wide variety of CNN model-families: (1) CNNs with fully-connected layers (e.g.VGG), (2) CNNs used for structured outputs (e.g.captioning), (3) CNNs used in tasks with multi-modal inputs (e.g.visual question answering) or reinforcement learning, all without architectural changes or re-training. We combine Grad-CAM with existing fine-grained visualizations to create a high-resolution class-discriminative visualization, Guided Grad-CAM, and apply it to image classification, image captioning, and visual question answering (VQA) models, including ResNet-based architectures. In the context of image classification models, our visualizations (a) lend insights into failure modes of these models (showing that seemingly unreasonable predictions have reasonable explanations), (b) outperform previous methods on the ILSVRC-15 weakly-supervised localization task, (c) are robust to adversarial perturbations, (d) are more faithful to the underlying model, and (e) help achieve model generalization by identifying dataset bias. For image captioning and VQA, our visualizations show that even non-attention based models learn to localize discriminative regions of input image. We devise a way to identify important neurons through Grad-CAM and combine it with neuron names (Bau et al. in Computer vision and pattern recognition, 2017) to provide textual explanations for model decisions. Finally, we design and conduct human studies to measure if Grad-CAM explanations help users establish appropriate trust in predictions from deep networks and show that Grad-CAM helps untrained users successfully discern a ‘stronger’ deep network from a ‘weaker’ one even when both make identical predictions. Our code is available at https://github.com/ramprs/grad-cam/, along with a demo on CloudCV (Agrawal et al., in: Mobile cloud visual media computing, pp 265–290. Springer, 2015) (http://gradcam.cloudcv.org) and a video at http://youtu.be/COjUB9Izk6E.


On the one hand, graph neural networks #REFR are initially defined in the spectral space, which is inspired by the convolution operations defined on the image grid.

Decades ago, the emerging big graph data on the website had already attracted much concern of researchers #OTHEREFR , #OTHEREFR , #OTHEREFR , #OTHEREFR , #OTHEREFR .

The learned low-dimensional embeddings for nodes, edges, and subgraphs can be easily integrated with the machine learning algorithms like Logistic Regression, Random Forest, and Gradient Boosting Decision Trees to perform risk monitoring #OTHEREFR , link prediction #OTHEREFR , item recommendation #OTHEREFR , etc.

Recently, researchers interested in graph mining have been motivated by the great success of deep learning methods in computer vision #OTHEREFR and natural language processing #OTHEREFR .

The deep graph embedding methods can also be categorized into two kinds: graph convolutional networks #OTHEREFR and skip-gram models #OTHEREFR .

GCN #OTHEREFR simplifies the learning framework of #OTHEREFR and shows a successful application on semi-supervised node classification tasks.

GraphSAGE #OTHEREFR proposes an inductive learning paradigm on large-scale graphs, which samples subgraphs for each node.

Meanwhile, GAT #OTHEREFR introduces a learnable attention mechanism to impose different importances over neighbors.

The attention mechanism not only achieves better performance on several benchmarks but also presents an explainable result by the neighbor importances.

On the other hand, DeepWalk #OTHEREFR is the pioneering work to introduce the skip-gram models #OTHEREFR into graph representation learning with the sampled node sequences by random walks.

Temporal Aggregation And Propagation Graph Neural Networks For Dynamic Representation

Abstract: Temporal graphs exhibit dynamic interactions between nodes over continuous time, whose topologies evolve with time elapsing. The whole temporal neighborhood of nodes reveals the varying preferences of nodes. However, previous works usually generate dynamic representation with limited neighbors for simplicity, which results in both inferior performance and high latency of online inference. Therefore, in this paper, we propose a novel method of temporal graph convolution with the whole neighborhood, namely Temporal Aggregation and Propagation Graph Neural Networks (TAP-GNN). Specifically, we firstly analyze the computational complexity of the dynamic representation problem by unfolding the temporal graph in a message-passing paradigm. The expensive complexity motivates us to design the AP (aggregation and propagation) block, which significantly reduces the repeated computation of historical neighbors. The final TAP-GNN supports online inference in the graph stream scenario, which incorporates the temporal information into node embeddings with a temporal activation function and a projection layer besides several AP blocks. Experimental results on various real-life temporal networks show that our proposed TAP-GNN outperforms existing temporal graph methods by a large margin in terms of both predictive performance and online inference latency. Our code is available at https://github.com/doujiang-zheng/TAP-GNN.

Convolutional Neural Networks On Graphs With Fast Localized Spectral Filtering

Abstract: In this work, we are interested in generalizing convolutional neural networks (CNNs) from low-dimensional regular grids, where image, video and speech are represented, to high-dimensional irregular domains, such as social networks, brain connectomes or words' embedding, represented by graphs. We present a formulation of CNNs in the context of spectral graph theory, which provides the necessary mathematical background and efficient numerical schemes to design fast localized convolutional filters on graphs. Importantly, the proposed technique offers the same linear computational complexity and constant learning complexity as classical CNNs, while being universal to any graph structure. Experiments on MNIST and 20NEWS demonstrate the ability of this novel deep learning system to learn local, stationary, and compositional features on graphs.


However, the state of S2V-DQN only considers the adjacency matrix and the state of nodes, and simply follows greedy policy at every step, which results in subpar performance #REFR .

In general, these GNN-based approaches require either one-hot encoded vectors #OTHEREFR , #OTHEREFR , #OTHEREFR or node feature matrix #OTHEREFR , #OTHEREFR , #OTHEREFR , #OTHEREFR as initial embeddings, which can be smoothed by the neighborhood aggregation to carrier graph structure information.

However, recent studies have highlighted the inherent over-smoothing and information loss issues of GNNs #OTHEREFR - #OTHEREFR .

This oversmoothing problem can also affect solving combinatorial optimization problems, as the goal is to identify distinguished nodes that maximize the objective function.

Reinforcement Learning combined with GNNs is becoming increasingly popular for solving CO problems.

Most no-tably, #OTHEREFR proposed S2V-DQN, a GNN-based reinforcement learning approach, which uses the neighborhood aggregation to update the node representation and learns a greedy policy using Q-learning with the graph embedding vector.

As data size continuously increases, developing models that can scale to modern dataset sizes is imperative.

Significant improvement in scalability can be seen in GComb #OTHEREFR .

GComb uses a probabilistic greedy mechanism to predict the quality of the nodes by a trained GCN.

By evaluating the quality of nodes, GComb is able to prune those unlikely to be in the solution set, allowing it to generalize to large graphs of millions or billions of nodes while maintaining the performance of S2V-DQN on Maximum Coverage on the bipartite graph (MCP), Maximum Vertex Cover (MVC), and Influence Maximization (IM).

However, similar to S2V-DQN, it does not involve local search behavior resulting in its performance being marginally lower than S2V-DQN.

Rels-Dqn: A Robust And Efficient Local Search Framework For Combinatorial Optimization

Abstract: Combinatorial optimization (CO) aims to efficiently find the best solution to NP-hard problems ranging from statistical physics to social media marketing. A wide range of CO applications can benefit from local search methods because they allow reversible action over greedy policies. Deep Q-learning (DQN) using message-passing neural networks (MPNN) has shown promise in replicating the local search behavior and obtaining comparable results to the local search algorithms. However, the over-smoothing and the information loss during the iterations of message passing limit its robustness across applications, and the large message vectors result in memory inefficiency. Our paper introduces RELS-DQN, a lightweight DQN framework that exhibits the local search behavior while providing practical scalability. Using the RELS-DQN model trained on one application, it can generalize to various applications by providing solution values higher than or equal to both the local search algorithms and the existing DQN models while remaining efficient in runtime and memory.

Combinatorial Optimization With Graph Convolutional Networks And Guided Tree Search

Abstract: We present a learning-based approach to computing solutions for certain NP-hard problems. Our approach combines deep learning techniques with useful algorithmic elements from classic heuristics. The central component is a graph convolutional network that is trained to estimate the likelihood, for each vertex in a graph, of whether this vertex is part of the optimal solution. The network is designed and trained to synthesize a diverse set of solutions, which enables rapid exploration of the solution space via tree search. The presented approach is evaluated on four canonical NP-hard problems and five datasets, which include benchmark satisfiability problems and real social network graphs with up to a hundred thousand nodes. Experimental results demonstrate that the presented approach substantially outperforms recent work, generalizes across datasets, and scales to graphs that are orders of magnitude larger than those used during training.