Scripting languages are well suited to such tasks because researchers can focus on high-level needs and worry less about memory management, code efficiency, and other technical details #REFR .

These opportunities have motivated the creation of interdisciplinary training programs, courses, workshops, and tutorials to teach computing skills in a life-sciences context 6,2,@ 7,8-12 .

In some circumstances, it is sufficient for researchers to understand computing concepts and learn to use existing tools; in other settings, learning to write computer code is invaluable #OTHEREFR .

A 2011 survey of scientists from many disciplines (other than computer science) found that researchers spent 35% of their time, on average, writing code #OTHEREFR .

Computer programming makes it possible to complete tasks not supported by existing tools, interface with software libraries, adapt algorithms based on custom needs, tidy data, and more #OTHEREFR [16] #OTHEREFR .

In these applied scenarios, computer programs are often small #OTHEREFR , and the code might only be used for one particular project.

Python, a scripting language, has gained much acceptance among scientists #OTHEREFR and programming educators #OTHEREFR , perhaps due to its relatively simple syntax 19 and the availability of libraries supporting common tasks #OTHEREFR .

However, learning to program is a daunting challenge for many researchers.

Decades of research have sought to characterize common errors and identify effective ways for novices to learn programming skills #OTHEREFR ; much remains to be discovered.

Recent advances in artificial intelligence have shown promise for converting natural-language descriptions of programming tasks to functional code 31, #OTHEREFR .

The first such large language models (LLMs) fine-tuned to generate code that captured widespread interest were OpenAI's Codex and DeepMind's AlphaCode #OTHEREFR .

Many Bioinformatics Programming Tasks Can Be Automated With Chatgpt

Abstract: Computer programming is a fundamental tool for life scientists, allowing them to carry out many essential research tasks. However, despite a variety of educational efforts, learning to write code can be a challenging endeavor for both researchers and students in life science disciplines. Recent advances in artificial intelligence have made it possible to translate human-language prompts to functional code, raising questions about whether these technologies can aid (or replace) life scientists' efforts to write code. Using 184 programming exercises from an introductory-bioinformatics course, we evaluated the extent to which one such model-OpenAI's ChatGPT-can successfully complete basic-to moderatelevel programming tasks. On its first attempt, ChatGPT solved 139 (75.5%) of the exercises. For the remaining exercises, we provided natural-language feedback to the model, prompting it to try different approaches. Within 7 or fewer attempts, ChatGPT solved 179 (97.3%) of the exercises. These findings have important implications for life-sciences research and education. For many programming tasks, researchers no longer need to write code from scratch. Instead, machine-learning models may produce usable solutions. Instructors may need to adapt their pedagogical approaches and assessment techniques to account for these new capabilities that are available to the general public.

An Introduction To Programming For Bioscientists: A Python-Based Primer

Abstract: Computing has revolutionized the biological sciences over the past several decades, such that virtually all contemporary research in the biosciences utilizes computer programs. The computational advances have come on many fronts, spurred by fundamental developments in hardware, software, and algorithms. These advances have influenced, and even engendered, a phenomenal array of bioscience fields, including molecular evolution and bioinformatics; genome-, proteome-, transcriptome- and metabolome-wide experimental studies; structural genomics; and atomistic simulations of cellular-scale molecular assemblies as large as ribosomes and intact viruses. In short, much of post-genomic biology is increasingly becoming a form of computational biology. The ability to design and write computer programs is among the most indispensable skills that a modern researcher can cultivate. Python has become a popular programming language in the biosciences, largely because (i) its straightforward semantics and clean syntax make it a readily accessible first language; (ii) it is expressive and well-suited to object-oriented programming, as well as other modern paradigms; and (iii) the many available libraries and third-party toolkits extend the functionality of the core language into virtually every biological domain (sequence and structure analyses, phylogenomics, workflow management systems, etc.). This primer offers a basic introduction to coding, via Python, and it includes concrete examples and exercises to illustrate the language's usage and capabilities; the main text culminates with a final project in structural bioinformatics. A suite of Supplemental Chapters is also provided. Starting with basic concepts, such as that of a 'variable', the Chapters methodically advance the reader to the point of writing a graphical user interface to compute the Hamming distance between two DNA sequences.


West and O'Shea #REFR apply a convolutional long short term deep neural network (CLDNN), which significantly improves classification accuracy.

Traditional AMC can be divided into two categories: likelihood-based (LB) methods #OTHEREFR and f eature-based (FB) methods #OTHEREFR .

However, LB methods rely on prior knowledge about channel and signal.

FB methods select hand-crafted features, then conduct the classification using the machine learning algorithm, such as support vector machines #OTHEREFR and random forests #OTHEREFR . FB methods highly depend on expert knowledge.

O'Shea, Corgan, and Clanc #OTHEREFR pioneer a CNN model for AMC, initiating the application of deep learning in AMC. It outperforms traditional methods that rely on manual features.

A model based on LSTM is proposed by #OTHEREFR . Huang et al. #OTHEREFR apply GRU to classify the signals.

Moreover, the dual-stream structure of CNN-LSTM is proposed in #OTHEREFR to efficiently classify signals.

It uses the information of I/Q channel and amplitude/phase to achieve better performance.

I and Q channel are intergrated in #OTHEREFR to learn the correlations of signals in parallel. Furthermore, Huynh-The et al.

#OTHEREFR implement different asymmetric convolution kernels and skip connections to learn spatial correlations. Liang et al.

#OTHEREFR combine the attention mechanism and complex-valued neural network to better represent the signal.

Amc-Net: An Effective Network For Automatic Modulation Classification

Abstract: Automatic modulation classification (AMC) is a crucial stage in the spectrum management, signal monitoring, and control of wireless communication systems. The accurate classification of the modulation format plays a vital role in the subsequent decoding of the transmitted data. End-to-end deep learning methods have been recently applied to AMC, outperforming traditional feature engineering techniques. However, AMC still has limitations in low signal-to-noise ratio (SNR) environments. To address the drawback, we propose a novel AMC-Net that improves recognition by denoising the input signal in the frequency domain while performing multi-scale and effective feature extraction. Experiments on two representative datasets demonstrate that our model performs better in efficiency and effectiveness than the most current methods.

Deep Architectures For Modulation Recognition

Abstract: We survey the latest advances in machine learning with deep neural networks by applying them to the task of radio modulation recognition. Results show that radio modulation recognition is not limited by network depth and further work should focus on improving learned synchronization and equalization. Advances in these areas will likely come from novel architectures designed for these tasks or through novel training methods.


DoReFa-Net #REFR proposed to optimize the clipping value and the scaling factor of the uniform quantizers for weights and activations separately.

It was validated on image classification tasks under multiple bit-widths, but only with the rather simple AlexNet architecture.

Most QAT works quantize weights and activations simultaneously by optimizing the uniform quantization parameters #OTHEREFR , layer-wise or channel-wise mixed-precision quantization #OTHEREFR , or leveraging nonuniform quantization such as Logarithmic quantizer #OTHEREFR .

Most recent QAT works #OTHEREFR used "Straight-Through Estimator" (STE) #OTHEREFR to estimate the gradient of the nondifferentiable quantization function, while another work #OTHEREFR softened the linear quantization operation in order to match the true gradient with STE.

#OTHEREFR adopted a primitive quantizer design based on a uniform quantizer for gradients (without scaling and other optimization), and large performance drops were observed when training with low-bit gradients.

SBM #OTHEREFR adopted fixed-point 8-bit gradient quantization but only focused on improving the quantization schemes in the forward pass.

Metagrad: Adaptive Gradient Quantization With Hypernetworks

Abstract: A popular track of network compression approach is Quantization aware Training (QAT), which accelerates the forward pass during the neural network training and inference. However, not much prior efforts have been made to quantize and accelerate the backward pass during training, even though that contributes around half of the training time. This can be partly attributed to the fact that errors of low-precision gradients during backward cannot be amortized by the training objective as in the QAT setting. In this work, we propose to solve this problem by incorporating the gradients into the computation graph of the next training iteration via a hypernetwork. Various experiments on CIFAR-10 dataset with different CNN network architectures demonstrate that our hypernetwork-based approach can effectively reduce the negative effect of gradient quantization noise and successfully quantizes the gradients to INT4 with only 0.64 accuracy drop for VGG-16 on CIFAR-10.

Dorefa-Net: Training Low Bitwidth Convolutional Neural Networks With Low Bitwidth Gradients

Abstract: We propose DoReFa-Net, a method to train convolutional neural networks that have low bitwidth weights and activations using low bitwidth parameter gradients. In particular, during backward pass, parameter gradients are stochastically quantized to low bitwidth numbers before being propagated to convolutional layers. As convolutions during forward/backward passes can now operate on low bitwidth weights and activations/gradients respectively, DoReFa-Net can use bit convolution kernels to accelerate both training and inference. Moreover, as bit convolutions can be efficiently implemented on CPU, FPGA, ASIC and GPU, DoReFa-Net opens the way to accelerate training of low bitwidth neural network on these hardware. Our experiments on SVHN and ImageNet datasets prove that DoReFa-Net can achieve comparable prediction accuracy as 32-bit counterparts. For example, a DoReFa-Net derived from AlexNet that has 1-bit weights, 2-bit activations, can be trained from scratch using 6-bit gradients to get 46.1\% top-1 accuracy on ImageNet validation set. The DoReFa-Net AlexNet model is released publicly.


Although there is no firm answer concerning these rules #REFR , it is commonly admitted that the definition of a community relates to a difference in connection density between its interior and its boundary.

Nodes of these networks are often arranged in closely tight groups called communities.

These communities delineate the organizational supports of function, property, purpose or categories.

They thus highlight a structure of the network providing an organizational understanding behind the topology.

Formally, the goal is to identify a node partition of the nodes of the network.

A community structure is a partition of the vertices of a graph defined according rules structuring the vertex distribution.

The density of connection between nodes inside a community must be higher than the density of connection across communities.

Such community obtained by this method is called the topological community #OTHEREFR .

Community detection algorithms capture this difference of connection density for detecting communities in a network #OTHEREFR .

The quality of a community structure is evaluated by a measure assessing this partitioning rule.

A recognized standard is the modularity introduced by Newmman #OTHEREFR .

Chromatic Community Structure Detection

Abstract: The detection of community structure is probably one of the hottest trends in complex network research as it reveals the internal organization of people, molecules or processes behind social, biological or computer networks.. . The issue is to provide a network partition representative of this organization so that each community presumably gathers nodes sharing a common mission, purpose or property. Usually the identification is based on the difference between the connectivity density of the interior and the boundary of a community. Indeed, nodes sharing a common purpose or property are expected to interact closely. Although this rule appears mostly relevant, some fundamental scientific problems like disease module detection highlight the inability to determine significantly the communities under this connectivity rule. The main reason is that the connectivity density is not correlated to a shared property or purpose. Therefore, another paradigm is required for properly formalize this issue in order to meaningfully detect these communities. In this article we study the community formation from this new principle. Considering colors formally figures the shared properties, the issue is thus to maximize group of nodes with the same color within communities.. We study this novel community framework by introducing new measurement called chromarity assessing the quality of the community structure regarding this constraint. Next we propose an algorithm solving the community structure detection based on this new community formation paradigm.

Community Detection In Networks: A User Guide

Abstract: Community detection in networks is one of the most popular topics of modern network science. Communities, or clusters, are usually groups of vertices having higher probability of being connected to each other than to members of other groups, though other patterns are possible. Identifying communities is an ill-defined problem. There are no universal protocols on the fundamental ingredients, like the definition of community itself, nor on other crucial issues, like the validation of algorithms and the comparison of their performances. This has generated a number of confusions and misconceptions, which undermine the progress in the field. We offer a guided tour through the main aspects of the problem. We also point out strengths and weaknesses of popular methods, and give directions to their use.


Experimental Setup 5.1 Experimental Settings

SASRec #REFR is trained with adam optimizer and a learning rate of 0.001. The number of layers and attention heads is 2.

For sequential recommendation baselines, we keep a consistent batch size of 4096 and the maximum interaction length to be 50.

We employ a leave-one-out strategy for validation and testing, and the full item set is used for evaluation. We use early stopping patience of 10.

We consider the sequential recommendation task as a multiclass classification task and use the cross-entropy loss for training the models.

The dropout rate is 0.5 and gelu is used as an activation function.

HGN #OTHEREFR and SINE #OTHEREFR are trained with embedding size 64, learning rate 0.001, and adam optimizer.

LigthSANs #OTHEREFR uses the latent interest dimension to be 5, 2 attention heads and 2 transformer layers.

Training is done with a learning rate of 0.001 and adam optimizer.

GCSAN #OTHEREFR has 2 transformer encoder layers, 2 attention heads, 64 is the hidden state features size, feed-forward layers' hidden size is 256, weight is set to be 0.6 and the number of layers in graph neural network is 1, adam is used for optimization with a learning rate of 0.001.

Mobilerec: A Large-Scale Dataset For Mobile Apps Recommendation

Abstract: Recommender systems have become ubiquitous in our digital lives, from recommending products on e-commerce websites to suggesting movies and music on streaming platforms. Existing recommendation datasets, such as Amazon Product Reviews and MovieLens, greatly facilitated the research and development of recommender systems in their respective domains. While the number of mobile users and applications (aka apps) has increased exponentially over the past decade, research in mobile app recommender systems has been significantly constrained, primarily due to the lack of high-quality benchmark datasets, as opposed to recommendations for products, movies, and news. To facilitate research for app recommendation systems, we introduce a large-scale dataset, called MobileRec. We constructed MobileRec from users' activity on the Google play store. MobileRec contains 19.3 million user interactions (i.e., user reviews on apps) with over 10K unique apps across 48 categories. MobileRec records the sequential activity of a total of 0.7 million distinct users. Each of these users has interacted with no fewer than five distinct apps, which stands in contrast to previous datasets on mobile apps that recorded only a single interaction per user. Furthermore, MobileRec presents users' ratings as well as sentiments on installed apps, and each app contains rich metadata such as app name, category, description, and overall rating, among others. We demonstrate that MobileRec can serve as an excellent testbed for app recommendation through a comparative study of several state-of-the-art recommendation approaches. The quantitative results can act as a baseline for other researchers to compare their results against.

Self-Attentive Sequential Recommendation

Abstract: Sequential dynamics are a key feature of many modern recommender systems, which seek to capture the 'context' of users' activities on the basis of actions they have performed recently. To capture such patterns, two approaches have proliferated: Markov Chains (MCs) and Recurrent Neural Networks (RNNs). Markov Chains assume that a user's next action can be predicted on the basis of just their last (or last few) actions, while RNNs in principle allow for longer-term semantics to be uncovered. Generally speaking, MC-based methods perform best in extremely sparse datasets, where model parsimony is critical, while RNNs perform better in denser datasets where higher model complexity is affordable. The goal of our work is to balance these two goals, by proposing a self-attention based sequential model (SASRec) that allows us to capture long-term semantics (like an RNN), but, using an attention mechanism, makes its predictions based on relatively few actions (like an MC). At each time step, SASRec seeks to identify which items are 'relevant' from a user's action history, and use them to predict the next item. Extensive empirical studies show that our method outperforms various state-of-the-art sequential models (including MC/CNN/RNN-based approaches) on both sparse and dense datasets. Moreover, the model is an order of magnitude more efficient than comparable CNN/RNN-based models. Visualizations on attention weights also show how our model adaptively handles datasets with various density, and uncovers meaningful patterns in activity sequences.


Its proof is the same as the proof of #REFR Lemma 4 .28], which directly translates to the locally finite case.

If there exists N such that s ′ ∈ Y ′ N then for all n N we must have s ′ ∈ Y n by definition of .

Up to extracting another infinite subsequence, we can assume that either s ′ ∈ Y ′ n for all n or s ′ ∈ Z ′ n for all n. As a result,

n∈N is an infinite decreasing sequence of separations of order 3 in T , contradicting the fact that T is a well-quasi-ordered set.

We conclude this subsection with the following result relating the degeneracy of minimal separations in G and G ′ .

To be more precise, we also need the additional assumption that T ′ is a region tangle to make the proof work, which is given by Lemma 3.25.

Lemma 3.26 (Lemma 4.28 and Corollary 4.29 in #OTHEREFR ).

Either G ′ is 4-connected and T ′ min = {(∅, ∅, V (G ′ ))}, or T ′ min = {(Y, S, Z) ∨ : (Y, S, Z) ∈ T min and S ∨ is a separator of G ′ }.

In the latter case, for all (Y, S, Z) ∈ T min , (Y, S, Z) is non-degenerate if and only if (Y, S, Z) ∨ is non-degenerate.

The Structure Of Quasi-Transitive Graphs Avoiding A Minor With Applications To The Domino Problem

Abstract: An infinite graph is quasi-transitive if its vertex set has finitely many orbits under the action of its automorphism group. In this paper we obtain a structure theorem for locally finite quasi-transitive graphs avoiding a minor, which is reminiscent of the Robertson-Seymour Graph Minor Structure Theorem. We prove that every locally finite quasi-transitive graph G avoiding a minor has a treedecomposition whose torsos are finite or planar; moreover the tree-decomposition is canonical, i.e. invariant under the action of the automorphism group of G. As applications of this result, we prove the following. • Every locally finite quasi-transitive graph attains its Hadwiger number, that is, if such a graph contains arbitrarily large clique minors, then it contains an infinite clique minor. This extends a result of Thomassen (1992) who proved it in the 4-connected case and suggested that this assumption could be omitted. In particular, this shows that a Cayley graph excludes a finite minor if and only if it avoids the countable clique as a minor. • Locally finite quasi-transitive graphs avoiding a minor are accessible (in the sense of Thomassen and Woess), which extends known results on planar graphs to any proper minor-closed family. • Minor-excluded finitely generated groups are accessible (in the group-theoretic sense) and finitely presented, which extends classical results on planar groups. • The domino problem is decidable in a minor-excluded finitely generated group if and only if the group is virtually free, which proves the minorexcluded case of a conjecture of Ballier and Stein (2018).

Quasi-4-Connected Components

Abstract: We introduce a new decomposition of a graphs into quasi-4-connected components, where we call a graph quasi-4-connected if it is 3-connected and it only has separations of order 3 that remove a single vertex. Moreover, we give a cubic time algorithm computing the decomposition of a given graph. Our decomposition into quasi-4-connected components refines the well-known decompositions of graphs into biconnected and triconnected components. We relate our decomposition to Robertson and Seymour's theory of tangles by establishing a correspondence between the quasi-4-connected components of a graph and its tangles of order 4.


This is in line with other studies in the image community #REFR . Chung et al.

As SSL models are becoming mainstream, many researchers have tried to understand what is happening under the hood of these pre-trained models either by learning a probing tasks #OTHEREFR or analyzing the representations directly #OTHEREFR are two most popular approaches.

The authors of #OTHEREFR study the WAV2VEC2.0 model #OTHEREFR representations directly rather than training additional classifiers as probes.

They show that the pre-trained model follows an auto-encoder style behavior i.e., intermediate layers provide richer information of higherlevel classes (phone/word information) than the initial and last layers.

#OTHEREFR studies the similarity between the representations of three SSL pre-training techniques i.e., contrastive predictive coding (CPC), auto-regressive predictive coding (APC), and masked predictive coding (MPC).

Their findings suggest that it is the learning objective which controls the representation similarity than architectural choices such as building blocks and directionality.

Similarly, in a comprehensive review of self-supervised speech representation learning by #OTHEREFR

(2022) #OTHEREFR , the authors posited that the choice of training criterion has a greater impact on the performance gains compared to the architecture or the directionality of input.

Given pre-training criterion has the highest impact on the model's performance, in this paper we investigate the impact of the masked prediction loss on the encoded information type in various layers of HuBERT model #OTHEREFR .

Analysing The Masked Predictive Coding Training Criterion For Pre-Training A Speech Representation Model

Abstract: Recent developments in pre-trained speech representation utilizing self-supervised learning (SSL) have yielded exceptional results on a variety of downstream tasks. One such technique, known as masked predictive coding (MPC), has been employed by some of the most high-performing models. In this study, we investigate the impact of MPC loss on the type of information learnt at various layers in the HuBERT model, using nine probing tasks. Our findings indicate that the amount of content information learned at various layers of the HuBERT model has a positive correlation to the MPC loss. Additionally, it is also observed that any speaker-related information learned at intermediate layers of the model, is an indirect consequence of the learning process, and therefore cannot be controlled using the MPC loss. These findings may serve as inspiration for further research in the speech community, specifically in the development of new pre-training tasks or the exploration of new pre-training criterion's that directly preserves both speaker and content information at various layers of a learnt model.

A Simple Framework For Contrastive Learning Of Visual Representations

Abstract: This paper presents SimCLR: a simple framework for contrastive learning of visual representations. We simplify recently proposed contrastive self-supervised learning algorithms without requiring specialized architectures or a memory bank. In order to understand what enables the contrastive prediction tasks to learn useful representations, we systematically study the major components of our framework. We show that (1) composition of data augmentations plays a critical role in defining effective predictive tasks, (2) introducing a learnable nonlinear transformation between the representation and the contrastive loss substantially improves the quality of the learned representations, and (3) contrastive learning benefits from larger batch sizes and more training steps compared to supervised learning. By combining these findings, we are able to considerably outperform previous methods for self-supervised and semi-supervised learning on ImageNet. A linear classifier trained on self-supervised representations learned by SimCLR achieves 76.5% top-1 accuracy, which is a 7% relative improvement over previous state-of-the-art, matching the performance of a supervised ResNet-50. When fine-tuned on only 1% of the labels, we achieve 85.8% top-5 accuracy, outperforming AlexNet with 100X fewer labels.


The larger improvement in scores for intent classification may be correlated with the fact that for our data augmentation experiment on paraphrasing and machine translation, we were only able to Table 6 : Accuracy results for intent classification on the validation set. Baseline: mBERT #REFR .

Our findings regarding the performance of larger models are also observed in the test set.

Table 6 presents the evaluation results for both slot filling and intent classification tasks across all three target languages. Our mT0 models strongly outperform the baseline models.

Specifically, our mT0 models outperformed the baseline models in all target languages for the intent classification task, highlighting the effectiveness of larger models for intent classification.

Moreover, our mT0 models also outperform the baseline models in two of the target languages for slot filling task, further indicating the superiority of larger models for sentence-level classification tasks.

The improvement in scores for intent classification is more evident than for slot filling.

LS: LASER #OTHEREFR +sBERT #OTHEREFR . LL: LASER+LaBSE #OTHEREFR . LX: LASER+XLM-R #OTHEREFR . Underline: Best-performing models for each setting.

Bold: Best F 1 score across all the experiments and settings.

augment data for intent classification, resulting in a larger improvement in performance for this task compared to slot filling.

It is worth noting that we use the validation set for model selection, which resulted in higher scores than those achieved on the test set.

This is because the validation data is similar to the data used during training, while the test data is entirely new and unseen.

Zero-Shot Slot And Intent Detection In Low-Resource Languages

Abstract: Intent detection and slot filling are critical tasks in spoken and natural language understanding for task-oriented dialog systems. In this work we describe our participation in the slot and intent detection for low-resource language varieties (SID4LR; Aepli et al. (2023)). We investigate the slot and intent detection (SID) tasks using a wide range of models and settings. Given the recent success of multitaskprompted finetuning of large language models, we also test the generalization capability of the recent encoder-decoder model mT0 (Muennighoff et al., 2022) on new tasks (i.e., SID) in languages they have never intentionally seen. We show that our best model outperforms the baseline by a large margin (up to +30 F 1 points) in both SID tasks.

Bert: Pre-Training Of Deep Bidirectional Transformers For Language Understanding

Abstract: We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications. ::: BERT is conceptually simple and empirically powerful. It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5% (7.7% point absolute improvement), MultiNLI accuracy to 86.7% (4.6% absolute improvement), SQuAD v1.1 question answering Test F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1 (5.1 point absolute improvement).

intent classification
machine translation


First we have found that MALA with subgradient clearly has the worst performance among all the methods, a finding agreeing with #REFR .

Next we test the algorithms with the additional Metropolis (accept-reject) step included.

More precisely we implement the following algorithms: MALA with subgradient, the PMALA method in #OTHEREFR , a variant of PMALA with Chambolle2004 replaced by K-step Chambolle-Pock, and the proposed PDFP based algorithm denoted as MALA-PDFP.

The results of all the methods are compared in Table 6 .3, and we reinstate that thanks to the Metropolis step, the samples are asymptotically unbiased.

For the stability of PMALA and MALA-PDFP, step size δ should be no larger than parameter ρ.

Following #OTHEREFR we fix δ = ρ and the values of them (that are shown in Table 6 .3) are chosen such that the acceptance rates of all the algorithms are around 50% [31, #OTHEREFR for fair comparison.

Moreover, in both MALA-PDFP and PMALA-CP, we can see that the results of K = 5 are rather close to those of K = 100 and PMALA where in both cases the subproblem is solved accurately.

Notably in Table 6 .3 the run time of MALA-PDFP for K = 100 is similar or less than that for K = 5, this is because in this experiments ρ is much smaller than Table 6 .2 and the stopping criteria x n,k+1 − x n,k < 10 −5 is met even k < 5.

More interestingly, however, PMALA-CP with K = 1 yields substantially worse results (in terms of ESS and ESJD) than the algorithms that solve the subprobem accurately, while MALA-PDFP with K = 1 produces results that are comparable to those.

While this is an interesting indicator that the 1-step MALA-PDFP may be an effective and efficient sampling algorithm, further investigation and more comprehensive tests of the method are needed. Table 6 .3: Comparison of the Metropolis-adjusted Langevin algorithms.

Approximate Primal-Dual Fixed-Point Based Langevin Algorithms For Non-Smooth Convex Potentials *

Abstract: The Langevin algorithms are frequently used to sample the posterior distributions in Bayesian inference. In many practical problems, however, the posterior distributions often consist of non-differentiable components, posing challenges for the standard Langevin algorithms, as they require to evaluate the gradient of the energy function in each iteration. To this end, a popular remedy is to utilize the proximity operator, and as a result one needs to solve a proximity subproblem in each iteration. The conventional practice is to solve the subproblems accurately, which can be exceedingly expensive, as the subproblem needs to be solved in each iteration. We propose an approximate primal-dual fixed-point algorithm for solving the subproblem, which only seeks an approximate solution of the subproblem and therefore reduces the computational cost considerably. We provide theoretical analysis of the proposed method and also demonstrate its performance with numerical examples.

Proximal Markov Chain Monte Carlo Algorithms

Abstract: This paper presents a new Metropolis-adjusted Langevin algorithm (MALA) that uses convex analysis to simulate efficiently from high-dimensional densities that are log-concave, a class of probability distributions that is widely used in modern high-dimensional statistics and data analysis. The method is based on a new first-order approximation for Langevin diffusions that exploits log-concavity to construct Markov chains with favourable convergence properties. This approximation is closely related to Moreau-Yoshida regularisations for convex functions and uses proximity mappings instead of gradient mappings to approximate the continuous-time process. The proposed method complements existing MALA methods in two ways. First, the method is shown to have very robust stability properties and to converge geometrically for many target densities for which other MALA are not geometric, or only if the step size is sufficiently small. Second, the method can be applied to high-dimensional target densities that are not continuously differentiable, a class of distributions that is increasingly used in image processing and machine learning and that is beyond the scope of existing MALA and HMC algorithms. To use this method it is necessary to compute or to approximate efficiently the proximity mappings of the logarithm of the target density. For several popular models, including many Bayesian models used in modern signal and image processing and machine learning, this can be achieved with convex optimisation algorithms and with approximations based on proximal splitting techniques, which can be implemented in parallel. The proposed method is demonstrated on two challenging high-dimensional and non-differentiable models related to image resolution enhancement and low-rank matrix estimation that are not well addressed by existing MCMC methodology.


Recent advances in differential rendering have enabled learning of shapes, as well as other scene properties such as appearance, only from images and without the need for 3D supervision #REFR .

coordinate-based multi-layer perceptrons #OTHEREFR , have become a popular method for reconstruction in recent years.

These methods encode continuous functions that model various scene properties, such as Signed Distance #OTHEREFR , radiance #OTHEREFR , and occupancy #OTHEREFR .

Variations of these include hybrid discrete-continuous representations that employ an external data structure, i.e.

a grid or an octree, to partition the implicit function #OTHEREFR .

The encoded shape can then be extracted via sphere tracing #OTHEREFR after querying the implicit function repeatedly.

Our approach falls into the paradigm of using neural fields for articulated object reconstruction and further learns a complete system for detection, pose estimation, and articulated shape reconstruction from a single observation.

Implicit Reconstruction of Non-Rigid Objects: Going beyond static scenes with rigid objects, #OTHEREFR handle dynamic scenes while #OTHEREFR focus on reconstructing humans by leveraging their strong shape and kinematic prior as well as the amount of readily available datasets.

[42] propose a general reconstruction framework to reconstruct any nonrigid entity (i.e.

humans, animals, or objects) given only an RGB-video without requiring a category-specific shape template, while #OTHEREFR focus on point cloud input data and split the prediction into a canonical shape and a deformation field.

One downside of general reconstruction methods is that they do not leverage the rigidity and kinematic constraints of articulated objects.

Carto: Category And Joint Agnostic Reconstruction Of Articulated Objects

Abstract: Figure 1. Visualization of CARTO on unseen object instances. We first use CARTO to jointly detect all objects in the scene and then articulate them while keeping the predicted shape code constant.

Nerf: Representing Scenes As Neural Radiance Fields For View Synthesis

Abstract: We present a method that achieves state-of-the-art results for synthesizing novel views of complex scenes by optimizing an underlying continuous volumetric scene function using a sparse set of input views. Our algorithm represents a scene using a fully-connected (non-convolutional) deep network, whose input is a single continuous 5D coordinate (spatial location $(x,y,z)$ and viewing direction $(\theta, \phi)$) and whose output is the volume density and view-dependent emitted radiance at that spatial location. We synthesize views by querying 5D coordinates along camera rays and use classic volume rendering techniques to project the output colors and densities into an image. Because volume rendering is naturally differentiable, the only input required to optimize our representation is a set of images with known camera poses. We describe how to effectively optimize neural radiance fields to render photorealistic novel views of scenes with complicated geometry and appearance, and demonstrate results that outperform prior work on neural rendering and view synthesis. View synthesis results are best viewed as videos, so we urge readers to view our supplementary video for convincing comparisons.


For example, we used Stanza #REFR to filter some web-scraped data for those containing an animate noun marked as an indirect object and provided this to the linguists.

• 1,000 single animate noun AGME • 500 single pronoun AGME • 500 with two or more AGMEs Linguists were given details of the various categories and attributes listed in section A and asked to find sentences such that each such category is well represented (depending on the relative ease of finding such sentences).

Linguists were also asked to prioritize diversity of animate nouns where possible.

They were allowed to pull examples sentences from natural text or construct them from scratch as they saw fit.

However, except for a small number of toy examples, we asked that they include only sentences that were natural in both English and their target language, and could reasonably appear in some imaginable context.

We provided samples of web-scraped data that had been filtered with various heuristics to help identify sentences fitting some of the harder-tosatisfy criteria.

In some cases these sentences were used directly, and in others they were modified slightly to fit the requirements.

Throughout the process, we prioritized diversity of sentence structure, domain and vocabulary.

Rather than produce a representative sample, our intention was to produce a corpus that would chal-lenge any tested systems on a wide range of phenomena.

Gate: A Challenge Set For Gender-Ambiguous Translation Examples

Abstract: Although recent years have brought significant progress in improving translation of unambiguously gendered sentences, translation of ambiguously gendered input remains relatively unexplored. When source gender is ambiguous, machine translation models typically default to stereotypical gender roles, perpetuating harmful bias. Recent work has led to the development of "gender rewriters" that generate alternative gender translations on such ambiguous inputs, but such systems are plagued by poor linguistic coverage. To encourage better performance on this task we present and release GATE, a linguistically diverse corpus of gender-ambiguous source sentences along with multiple alternative target language translations. We also provide tools for evaluation and system analysis when using GATE and use them to evaluate our translation rewriter system.

Stanza: A Python Natural Language Processing Toolkit For Many Human Languages

Abstract: We introduce Stanza, an open-source Python natural language processing toolkit supporting 66 human languages. Compared to existing widely used toolkits, Stanza features a language-agnostic fully neural pipeline for text analysis, including tokenization, multi-word token expansion, lemmatization, part-of-speech and morphological feature tagging, dependency parsing, and named entity recognition. We have trained Stanza on a total of 112 datasets, including the Universal Dependencies treebanks and other multilingual corpora, and show that the same neural architecture generalizes well and achieves competitive performance on all languages tested. Additionally, Stanza includes a native Python interface to the widely used Java Stanford CoreNLP software, which further extends its functionalities to cover other tasks such as coreference resolution and relation extraction. Source code, documentation, and pretrained models for 66 languages are available at https://stanfordnlp.github.io/stanza.


Although directly applying this algorithm can achieve good results, it is incompatible with SAM due to the varying scales of gradient norm #REFR .

Once the update for every model is complete, we calculate the final update using a larger step size α as

x t+1 = clip xnat, (x t + α • (x n t − x t ))

To solve this problem, we normalize the gradient at each update by their 2 norm.

We discover that the modified version actually maximizes the cosine similarity between gradients (proof in Appendix E).

Thus, we call it Cosine Similarity Encourager (CSE), which can be further combined with MI as MI-CSE.

MI-CSE involves an inner momentum term to accumulate the gradients of each model. We provide the pseudocode in Appendix E.

Rethinking Model Ensemble In Transfer-Based Adversarial Attacks

Abstract: Deep learning models are vulnerable to adversarial examples. Transfer-based adversarial attacks attract tremendous attention as they can identify the weaknesses of deep learning models in a black-box manner. An effective strategy to improve the transferability of adversarial examples is attacking an ensemble of models. However, previous works simply average the outputs of different models, lacking an in-depth analysis on how and why model ensemble can strongly improve the transferability. In this work, we rethink the ensemble in adversarial attacks and define the common weakness of model ensemble with the properties of the flatness of loss landscape and the closeness to the local optimum of each model. We empirically and theoretically show that these two properties are strongly correlated with the transferability and propose a Common Weakness Attack (CWA) to generate more transferable adversarial examples by promoting these two properties. Experimental results on both image classification and object detection tasks validate the effectiveness of our approach to improve the adversarial transferability, especially when attacking adversarially trained models.

Boosting Adversarial Attacks With Momentum

Abstract: Deep neural networks are vulnerable to adversarial examples, which poses security concerns on these algorithms due to the potentially severe consequences. Adversarial attacks serve as an important surrogate to evaluate the robustness of deep learning models before they are deployed. However, most of existing adversarial attacks can only fool a black-box model with a low success rate. To address this issue, we propose a broad class of momentum-based iterative algorithms to boost adversarial attacks. By integrating the momentum term into the iterative process for attacks, our methods can stabilize update directions and escape from poor local maxima during the iterations, resulting in more transferable adversarial examples. To further improve the success rates for black-box attacks, we apply momentum iterative algorithms to an ensemble of models, and show that the adversarially trained models with a strong defense ability are also vulnerable to our black-box attacks. We hope that the proposed methods will serve as a benchmark for evaluating the robustness of various deep models and defense methods. With this method, we won the first places in NIPS 2017 Non-targeted Adversarial Attack and Targeted Adversarial Attack competitions.


Class Imbalance Our dataset is more balanced on the binary label (24% sexism) than many previous datasets for hate and abuse detection #REFR .

However, there is substantial imbalance in categories and vectors of sexism.

Thus, it is hard to confirm whether confusion between categories and vectors is due to inherent features of the data or due to the class imbalance.

We encourage future work to examine the effect on performance and cross-vector confusion when balancing the dataset.

That said, we explicitly made the decision to not re-balance our dataset because different types of sexism (especially at the finegrained level) do have very different base rates 'in the wild'.

Dealing with class imbalance is then part-and-parcel of the problem we seek to address.

Semeval-2023 Task 10: Explainable Detection Of Online Sexism

Abstract: Online sexism is a widespread and harmful phenomenon. Automated tools can assist the detection of sexism at scale. Binary detection, however, disregards the diversity of sexist content, and fails to provide clear explanations for why something is sexist. To address this issue, we introduce SemEval Task 10 on the Explainable Detection of Online Sexism (EDOS). We make three main contributions: i) a novel hierarchical taxonomy of sexist content, which includes granular vectors of sexism to aid explainability; ii) a new dataset of 20,000 social media comments with fine-grained labels, along with larger unlabelled datasets for model adaptation; and iii) baseline models as well as an analysis of the methods, results and errors for participant submissions to our task. Content warning: We show illustrative examples of sexist language to describe the taxonomy and analyse error types.

Directions In Abusive Language Training Data: Garbage In, Garbage Out

Abstract: Data-driven analysis and detection of abusive online content covers many different tasks, phenomena, contexts, and methodologies. This paper systematically reviews abusive language dataset creation and content in conjunction with an open website for cataloguing abusive language data. This collection of knowledge leads to a synthesis providing evidence-based recommendations for practitioners working with this complex and highly diverse data.


DreamerV2 is composed of the following components: an image encoder ∼ ( | , ℎ ) that learns a posterior latent representation conditional on the current observation and recurrent state ℎ , the recurrent state space model (RSSM) #REFR which is a mixture of deterministic and stochastic categorical latents, and the image, reward and discount predictors.

To learn a world model for behaviour learning and look-ahead shielding we leverage DreamerV2 #OTHEREFR , which was used to master Atari games in the ALE #OTHEREFR .

The RSSM consists of two main components: the recurrent model

, which computes the next deterministic latents given the past state −1 = (ℎ −1 , −1 ) and action −1 , and the transition predictorˆ∼ (ˆ| ℎ ), which is used as the prior distribution over the stochastic latents conditional on the deterministic latents.

The image predictor or decoderˆ∼ (ˆ| ℎ , ) is trained to predict the current observation with a reconstruction loss.

The image predictor provides useful self-supervised gradients that help the world model learn a structured latent space for effective policy optimisation #OTHEREFR .

Approximate Shielding Of Atari Agents For Safe Exploration

Abstract: Balancing exploration and conservatism in the constrained setting is an important problem if we are to use reinforcement learning for meaningful tasks in the real world. In this paper, we propose a principled algorithm for safe exploration based on the concept of shielding. Previous approaches to shielding assume access to a safety-relevant abstraction of the environment or a high-fidelity simulator. Instead, our work is based on latent shielding-another approach that leverages world models to verify policy roll-outs in the latent space of a learned dynamics model. Our novel algorithm builds on this previous work, using safety critics and other additional features to improve the stability and farsightedness of the algorithm. We demonstrate the effectiveness of our approach by running experiments on a small set of Atari games with state dependent safety labels. We present preliminary results that show our approximate shielding algorithm effectively reduces the rate of safety violations, and in some cases improves the speed of convergence and quality of the final agent.

Learning Latent Dynamics For Planning From Pixels

Abstract: Planning has been very successful for control tasks with known environment dynamics. To leverage planning in unknown environments, the agent needs to learn the dynamics from interactions with the world. However, learning dynamics models that are accurate enough for planning has been a long-standing challenge, especially in image-based domains. We propose the Deep Planning Network (PlaNet), a purely model-based agent that learns the environment dynamics from images and chooses actions through fast online planning in latent space. To achieve high performance, the dynamics model must accurately predict the rewards ahead for multiple time steps. We approach this using a latent dynamics model with both deterministic and stochastic transition components. Moreover, we propose a multi-step variational inference objective that we name latent overshooting. Using only pixel observations, our agent solves continuous control tasks with contact dynamics, partial observability, and sparse rewards, which exceed the difficulty of tasks that were previously solved by planning with learned models. PlaNet uses substantially fewer episodes and reaches final performance close to and sometimes higher than strong model-free algorithms.

stochastic categorical latents
posterior latent representation


The total variation of a function is defined as |Du|(Ω) = sup{ Ω u∇ • gdx : g ∈ (C 1 0 (Ω)) #REFR and |g(x)| ≤ 1 in Ω}, which is not differentiable as L 2 and H 1 -norm.

Usually, the unknown conductivity σ is not smooth and does not lie in H 1 0 (Ω C ).

In this situation, the H 1 -regularization is not suitable.

For reconstructing nonsmooth parameter, it is known that total variation regularization is a good choice.

Comparing with H 1 regularization, total regularization can deal with non-smooth parameters.

Here we give some reference about the total-variation regularization, such as #OTHEREFR .

This brings us difficulty in solving the optimization problem by iterative method based on gradient.

One way to deal with this difficulty is replacing T V term |Du| by its smoothness |∇u| 2 + ν 2 for small ν > 0 #OTHEREFR .

Another way is employing the splitting method #OTHEREFR .

To deal with the discontinuity of the parameter σ, we introduce the total variation regularization to the inverse eddy current problem.

In the present work, we treat the T V regularization with splitting method.

Iterative Methods For An Inverse Eddy Current Problem With Total Variation Regularization

Abstract: Conductivity reconstruction in an inverse eddy current problem is considered in the present paper. With the electric field measurement on part of domain boundary, we formulate the reconstruction problem to a constrained optimization problem with total variation regularization. Existence and stability are proved for the solution to the optimization problem. The finite element method is employed to discretize the optimization problem. The gradient Lipschitz properties of the objective functional are established for the the discrete optimization problems. We propose the alternating direction method of multipliers to solve the discrete problem. Based on the the gradient Lipschitz property, we prove the convergence by extending the admissible set to the whole finite element space. Finally, we show some numerical experiments to illustrate the efficiency of the proposed methods.

Detection And Classification From Electromagnetic Induction Data

Abstract: In this paper we introduce an efficient algorithm for identifying conductive objects using induction data derived from eddy currents. Our method consists of first extracting geometric features from the induction data and then matching them to precomputed data for known objects from a given dictionary. The matching step relies on fundamental properties of conductive polarization tensors and new invariants introduced in this paper. A new shape identification scheme is introduced and studied. We test it numerically in the presence of measurement noise. Stability and resolution capabilities of the proposed identification algorithm are quantified in numerical simulations.


Since the COCOCON is created from COCO Captions #REFR , the original captions predominantly contain animals defined in COCO objects.

Adjectives used as modifiers to a noun in the caption, such as color (red chair), height (tall building), size (small), and material (tiled wall).

• Food. Food-related concepts including fruits, vegetables, and cooked items.

Sport-related objects and references to the sport itself are included in this category.

• Animal. Includes all mentions of animals.

Includes broadly defined areas (e.g., bathroom, hotel, library), finer visual elements (e.g., floor, sidewalk), and spatial references (e.g., inside, outside, on table)

• Action. Comprises transitive (e.g. flying kite) as well as intransitive actions (e.g. sitting, standing) performed by persons and animals.

Concepts from one of the categories: man/male/guy, woman/female/lady, boy, girl.

Exposing And Addressing Cross-Task Inconsistency In Unified Vision-Language Models

Abstract: As general purpose vision models get increasingly effective at a wide set of tasks, it is imperative that they be consistent across the tasks they support. Inconsistent AI models are considered brittle and untrustworthy by human users and are more challenging to incorporate into larger systems that take dependencies on their outputs. Measuring consistency between very heterogeneous tasks that might include outputs in different modalities is challenging since it is difficult to determine if the predictions are consistent with one another. As a solution, we introduce a benchmark dataset, COCOCON, where we use contrast sets created by modifying test instances for multiple tasks in small but semantically meaningful ways to change the gold label, and outline metrics for measuring if a model is consistent by ranking the original and perturbed instances across tasks. We find that state-of-the-art systems suffer from a surprisingly high degree of inconsistent behavior across tasks, especially for more heterogeneous tasks. Finally, we propose using a rank correlation-based auxiliary objective computed over large automatically created cross-task contrast sets to improve the multi-task consistency of large unified models, while retaining their original accuracy on downstream tasks.

Microsoft Coco: Common Objects In Context

Abstract: We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding. This is achieved by gathering images of complex everyday scenes containing common objects in their natural context. Objects are labeled using per-instance segmentations to aid in precise object localization. Our dataset contains photos of 91 objects types that would be easily recognizable by a 4 year old. With a total of 2.5 million labeled instances in 328k images, the creation of our dataset drew upon extensive crowd worker involvement via novel user interfaces for category detection, instance spotting and instance segmentation. We present a detailed statistical analysis of the dataset in comparison to PASCAL, ImageNet, and SUN. Finally, we provide baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model.


Codag: Complementary Domain Adaptation And Generalization

To apply this approach to the SHOT algorithm #REFR with the current target domain D t , we first initialize the DA model f DA with the parameters of the DG model trained with previously experienced domains, or θ * DG,t−1 , treating it as a source model.

where θ DG,0 is the DG model parameters, L ce is the crossentropy loss for a classification setting, and R represents the RandMix augmentation #OTHEREFR .

Generalized initialization with DG for DA In general, unsupervised source-free domain adaptation approaches involve initially training a source model with the source domain data and further updating the source model with unlabeled data from a new target domain.

However, in our proposed framework, the DA model utilizes the parameters of the previous DG model for its initialization.

This allows the DA model to leverage the DG model's generalization ability to learn domain-invariant features and reduce domainspecific factors.

As a result, we achieve efficient adaptation to a new target domain, even when there is a large gap between previously experienced domains and the new target domain (see Section 4.2 for experimental results).

Then, we freeze the classifier head in f DA and only update the feature extractor part of it using information maximization and self-supervised pseudolabeling with data from the current target domain D t .

Accordingly, the loss to adapt to D t is written as,

where θ DA,t is the parameters of the DA model initialized with the optimal parameters of the DG model trained on the previous domain D t−1 , or θ * DG,t−1 , and L shot is the loss of the SHOT algorithm.

Pseudo-label generation with DA for DG In our proposed framework, we simply use an empirical risk minimization (ERM) method along with an enhanced data augmentation method for training the DG model.

Complementary Domain Adaptation And Generalization For Unsupervised Continual Domain Shift Learning

Abstract: Continual domain shift poses a significant challenge in real-world applications, particularly in situations where labeled data is not available for new domains. The challenge of acquiring knowledge in this problem setting is referred to as unsupervised continual domain shift learning. Existing methods for domain adaptation and generalization have limitations in addressing this issue, as they focus either on adapting to a specific domain or generalizing to unseen domains, but not both. In this paper, we propose Complementary Domain Adaptation and Generalization (CoDAG), a simple yet effective learning framework that combines domain adaptation and generalization in a complementary manner to achieve three major goals of unsupervised continual domain shift learning: adapting to a current domain, generalizing to unseen domains, and preventing forgetting of previously seen domains. Our approach is modelagnostic, meaning that it is compatible with any existing domain adaptation and generalization algorithms. We evaluate CoDAG on several benchmark datasets and demonstrate that our model outperforms state-of-the-art models in all datasets and evaluation metrics, highlighting its effectiveness and robustness in handling unsupervised continual domain shift learning.

Do We Really Need To Access The Source Data? Source Hypothesis Transfer For Unsupervised Domain Adaptation

Abstract: Unsupervised domain adaptation (UDA) aims to leverage the knowledge learned from a labeled source dataset to solve similar tasks in a new unlabeled domain. Prior UDA methods typically require to access the source data when learning to adapt the model, making them risky and inefficient for decentralized private data. In this work we tackle a novel setting where only a trained source model is available and investigate how we can effectively utilize such a model without source data to solve UDA problems. To this end, we propose a simple yet generic representation learning framework, named \emph{Source HypOthesis Transfer} (SHOT). Specifically, SHOT freezes the classifier module (hypothesis) of the source model and learns the target-specific feature extraction module by exploiting both information maximization and self-supervised pseudo-labeling to implicitly align representations from the target domains to the source hypothesis. In this way, the learned target model can directly predict the labels of target data. We further investigate several techniques to refine the network architecture to parameterize the source model for better transfer performance. To verify its versatility, we evaluate SHOT in a variety of adaptation cases including closed-set, partial-set, and open-set domain adaptation. Experiments indicate that SHOT yields state-of-the-art results among multiple domain adaptation benchmarks.

previously experienced domains
current target domain


Further, we also visualize the different components of the QoE metric from equation #REFR to understand how ALISA performs better than other ABR algorithms.

We have also compared ALISA with Pensieve, an RL-based ALISA achieves a higher QoE on all metrics over all different configurations.

ALISA obtains up to 25% higher QoE than RB, 230% higher QoE than BB, 30% higher QoE than BOLA, 25% higher QoE than RobustMPC and 20% higher QoE compared to Pensieve when tested under lossless conditions. This performance translates to lossy conditions as well.

We note that ALISA is able to obtain up to 25%, 28%, 48%, and 48% higher QoE compared to Pensieve under losses of 0.1%, 0.5%, 1%, and 2%, respectively.

We summarize the remained of our testing QoE metrics for a random packet loss percentage of 0.1%, 0.5%, 1%, and 2% in Table III, Table IV, Table V  and Table VI , respectively.

These results indicate that ALISA achieves a significantly better performance than many other fixed-rule-based ABR algorithms and also Pensieve.

Figure 5 presents the total reward achieved by various ABR algorithms with QoE lin metric for each trace when the network is emulated with 0.1% packet loss.

Our results show that the ALISA algorithm achieves a higher average QoE of 44.58 as compared to other ABR algorithms.

Figure 6 presents the average total reward achieved by various ABR algorithms with QoE lin metric for each trace when the network is emulated with 1% packet loss.

Our results show that the ALISA algorithm achieves a higher average QoE of 34.87 as compared to other ABR algorithms.

Figure 7 (a) and Figure 7 (b) shows how ALISA can consistently achieve higher bitrates than other methods for random sample traces. This increases the first component of QoE.

Deep Reinforcement Learning With Importance Weighted A3C For Qoe Enhancement In Video Delivery Services

Abstract: Adaptive bitrate (ABR) algorithms are used to adapt the video bitrate based on the network conditions to improve the overall video quality of experience (QoE). Recently, reinforcement learning (RL) and asynchronous advantage actor-critic (A3C) methods have been used to generate adaptive bit rate algorithms and they have been shown to improve the overall QoE as compared to fixed rule ABR algorithms. However, a common issue in the A3C methods is the lag between behaviour policy and target policy. As a result, the behaviour and the target policies are no longer synchronized which results in suboptimal updates. In this work, we present ALISA: An Actor-Learner Architecture with Importance Sampling for efficient learning in ABR algorithms. ALISA incorporates importance sampling weights to give more weightage to relevant experience to address the lag issues with the existing A3C methods. We present the design and implementation of ALISA, and compare its performance to state-of-the-art video rate adaptation algorithms including vanilla A3C implemented in the Pensieve framework and other fixed-rule schedulers like BB, BOLA, and RB. Our results show that ALISA improves average QoE by up to 25%-48% higher average QoE than Pensieve, and even more when compared to fixed-rule schedulers.

Asynchronous Methods For Deep Reinforcement Learning

Abstract: We propose a conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers. We present asynchronous variants of four standard reinforcement learning algorithms and show that parallel actor-learners have a stabilizing effect on training allowing all four methods to successfully train neural network controllers. The best performing method, an asynchronous variant of actor-critic, surpasses the current state-of-the-art on the Atari domain while training for half the time on a single multi-core CPU instead of a GPU. Furthermore, we show that asynchronous actor-critic succeeds on a wide variety of continuous motor control problems as well as on a new task of navigating random 3D mazes using a visual input.


To demonstrate the benefit of our approach we compare it against the following baselines: BPRMF #REFR Matrix factorisation optimised by the BPR loss function.

iALS #OTHEREFR matrix factorization learned by implicit alternating least squares.

PureSVD #OTHEREFR Compute item embeddings through a singular value decomposition of the user-item interaction matrix, which will be then used to infer user representations.

FISM #OTHEREFR Learn item embeddings through optimisation process creating user representations as a weighted combination of items in their profile.

Additional user and item biases as well as an agreement term are considered in the score estimation.

NGCF #OTHEREFR Work that introduces graph convolution to the collaborative filtering scenario, it uses dense layer and inner product to enrich the knowledge injected in the user item embeddings during the convolution process.

Item Graph Convolution Collaborative Filtering For Inductive Recommendations

Abstract: Graph Convolutional Networks (GCN) have been recently employed as core component in the construction of recommender system algorithms, interpreting user-item interactions as the edges of a bipartite graph. However, in the absence of side information, the majority of existing models adopt an approach of randomly initialising the user embeddings and optimising them throughout the training process. This strategy makes these algorithms inherently transductive, curtailing their ability to generate predictions for users that were unseen at training time. To address this issue, we propose a convolution-based algorithm, which is inductive from the user perspective, while at the same time, depending only on implicit user-item interaction data. We propose the construction of an item-item graph through a weighted projection of the bipartite interaction network and to employ convolution to inject higher order associations into item embeddings, while constructing user representations as weighted sums of the items with which they have interacted. Despite not training individual embeddings for each user our approach achieves state-of-the-art recommendation performance with respect to transductive baselines on four real-world datasets, showing at the same time robust inductive performance.

Bpr: Bayesian Personalized Ranking From Implicit Feedback

Abstract: Item recommendation is the task of predicting a personalized ranking on a set of items (e.g. websites, movies, products). In this paper, we investigate the most common scenario with implicit feedback (e.g. clicks, purchases). There are many methods for item recommendation from implicit feedback like matrix factorization (MF) or adaptive k-nearest-neighbor (kNN). Even though these methods are designed for the item prediction task of personalized ranking, none of them is directly optimized for ranking. In this paper we present a generic optimization criterion BPR-Opt for personalized ranking that is the maximum posterior estimator derived from a Bayesian analysis of the problem. We also provide a generic learning algorithm for optimizing models with respect to BPR-Opt. The learning method is based on stochastic gradient descent with bootstrap sampling. We show how to apply our method to two state-of-the-art recommender models: matrix factorization and adaptive kNN. Our experiments indicate that for the task of personalized ranking our optimization method outperforms the standard learning techniques for MF and kNN. The results show the importance of optimizing models for the right criterion.


For the adaptive entropy regularizer #REFR , we regularize policy to have target entropy −|A|, where the entropy regularizer weight is initialized to be 1 and optimized in log-space with a learning rate of 3 × 10 −4 .

For d θ , we use a 128-512-2048 projector followed by an IQE-maxmean head with 64 components, each of size 32.

We use x-512-512-8 network for policy, where x is the input size and 8 parametrizes a tanh-transformed diagonal Normal distribution. L transition is optimized with a weight of 0.1.

Our learning rates are 0.01 for λ, 1 × 10 −4 for the model parameters, and 3 × 10 −5 for the policy parameters. We use a batch size of 256 in training.

We prefill the replay buffer with 200 episodes from a random actor, and then iteratively perform (1) generating 10 rollouts and (2) optimizing QRL objective for 500 gradients steps.

We use N (0, 0.3 2 )-perturbed action noise in exploration.

Since the environment has much shorter horizon (each episodes ends at 50 timesteps), we instead use a different affine-transformed softplus for maximizing d θ , where φ(x) −softplus(15 − x, β = 0.1).

All settings are the same as QRL for state-based observations, except a few changes:

• We use the convolutional backbone followed by a x-512-128 network for encoder f .

• We optimize L transition with an increased weight of 10 (since the dynamics aren't fully deterministic).

Optimal Goal-Reaching Reinforcement Learning Via Quasimetric Learning

Abstract: In goal-reaching reinforcement learning (RL), the optimal value function has a particular geometry, called quasimetric structure. This paper introduces Quasimetric Reinforcement Learning (QRL), a new RL method that utilizes quasimetric models to learn optimal value functions. Distinct from prior approaches, the QRL objective is specifically designed for quasimetrics, and provides strong theoretical recovery guarantees. Empirically, we conduct thorough analyses on a discretized MountainCar environment, identifying properties of QRL and its advantages over alternatives. On offline and online goal-reaching benchmarks, QRL also demonstrates improved sample efficiency and performance, across both state-based and image-based observations.

Soft Actor-Critic Algorithms And Applications

Abstract: Model-free deep reinforcement learning (RL) algorithms have been successfully applied to a range of challenging sequential decision making and control tasks. However, these methods typically suffer from two major challenges: high sample complexity and brittleness to hyperparameters. Both of these challenges limit the applicability of such methods to real-world domains. In this paper, we describe Soft Actor-Critic (SAC), our recently introduced off-policy actor-critic algorithm based on the maximum entropy RL framework. In this framework, the actor aims to simultaneously maximize expected return and entropy. That is, to succeed at the task while acting as randomly as possible. We extend SAC to incorporate a number of modifications that accelerate training and improve stability with respect to the hyperparameters, including a constrained formulation that automatically tunes the temperature hyperparameter. We systematically evaluate SAC on a range of benchmark tasks, as well as real-world challenging tasks such as locomotion for a quadrupedal robot and robotic manipulation with a dexterous hand. With these improvements, SAC achieves state-of-the-art performance, outperforming prior on-policy and off-policy methods in sample-efficiency and asymptotic performance. Furthermore, we demonstrate that, in contrast to other off-policy algorithms, our approach is very stable, achieving similar performance across different random seeds. These results suggest that SAC is a promising candidate for learning in real-world robotics tasks.

learning rate
adaptive entropy regularizer


Significant research has also been done on quantization-aware training methods #REFR , where the loss in accuracy due to bit precision reduction is minimized.

Quantization aims to shrink the memory footprint of deep neural networks by reducing the number of bits used to store the values for the learnable parameters and activations.

This is not only ideal for application scenarios where memory resources may be restricted such as embedded systems or microcontroller environments, but with the selected weight representation you could potentially facilitate faster inference using cheaper arithmetic operations #OTHEREFR .

With the reduction in parameter bit precision, however, it is typical that a quantized neural network will perform worse in terms of accuracy than its non-quantized counterpart using gradient-based learning methods.

However, these drops in accuracy are usually considered minimal and worth the given benefit in memory reduction and inference speed-up.

There exists much literature describing various techniques of quantization and successful results thereof, including works utilizing stochastic rounding to select weight values beneficial to gradient training #OTHEREFR and applications on modern deep architectures #OTHEREFR , as well as quantization methods that reduce the number of multiplication operations required during training time #OTHEREFR .

Some of these quantization-aware training methods utilize a straightthrough gradient estimator (STE) #OTHEREFR to more appropriately select weights during network training that minimizes accuracy and further reduces computational burden #OTHEREFR .

As quantization methods are used for neural network reduction, there inevitably exist discrepancies between the performances of original and compressed neural networks.

In this work, we propose a computationally tractable approach to compute the guaranteed output error caused by quantization.

A merged neural network is constructed to generate the output differences between two neural networks, and then reachability analysis on the merged neural network can be performed to obtain the guaranteed error.

The remainder of the paper is organized as follows: Preliminaries are given in Section II.

Guaranteed Quantization Error Computation For Neural Network Model Compression

Abstract: Neural network model compression techniques can address the computation issue of deep neural networks on embedded devices in industrial systems. The guaranteed output error computation problem for neural network compression with quantization is addressed in this paper. A merged neural network is built from a feedforward neural network and its quantized version to produce the exact output difference between two neural networks. Then, optimization-based methods and reachability analysis methods are applied to the merged neural network to compute the guaranteed quantization error. Finally, a numerical example is proposed to validate the applicability and effectiveness of the proposed approach.

Value-Aware Quantization For Training And Inference Of Neural Networks

Abstract: We propose a novel value-aware quantization which applies aggressively reduced precision to the majority of data while separately handling a small amount of large data in high precision, which reduces total quantization errors under very low precision. We present new techniques to apply the proposed quantization to training and inference. The experiments show that our method with 3-bit activations (with 2% of large ones) can give the same training accuracy as full-precision one while offering significant (41.6% and 53.7%) reductions in the memory cost of activations in ResNet-152 and Inception-v3 compared with the state-of-the-art method. Our experiments also show that deep networks such as Inception-v3, ResNet-101 and DenseNet-121 can be quantized for inference with 4-bit weights and activations (with 1% 16-bit data) within 1% top-1 accuracy drop.


It also requires Lemma 3.7, the proof of which is adapted from that of #REFR Lemma 7] and is deferred to the Appendix.

, ℓ σ (v s ) ≤ pℓ} |C (G)| ≤ |C (G \ J)| • (pℓ) s |C (G \ J)| • ℓ s = p s .

The upper bound for the numerator in the above inequality comes from two facts that rely on J being an independent set.

First, given a colouring σ 0 ∈ C (G \ J), the number of extensions of σ 0 to a colouring σ ∈ C (G) is v∈J ℓ σ 0 (v).

Second, given such an extension σ, we have ℓ σ (v) = ℓ σ 0 (v) for every v ∈ J. We conclude that (X v ) v∈I are Ber(p)-dominated.

Leveraging Theorem 3.6, and the fact that neighbourhoods induce independent sets, we can prove the following exponential upper bound on the likelihood of short lists.

Let G be a triangle-free graph, let v ∈ V (G) and let σ 0 be a proper k-colouring of G \ N [v] (for some k), with at least one extension to G.

Then if σ is the uniformly random extension of σ 0 to G, writing

ℓ := E [ℓ σ (v)], we have P [ℓ σ (v) ≤ (1 − δ)ℓ] ≤ e − δ 2 ℓ 2 ,

Uniformly Random Colourings Of Sparse Graphs

Abstract: We analyse uniformly random proper k-colourings of sparse graphs with maximum degree ∆ in the regime ∆ < k ln k. This regime corresponds to the lower side of the shattering threshold for random graph colouring, a paradigmatic example of the shattering threshold for random Constraint Satisfaction Problems. We prove a variety of results about the solution space geometry of colourings of fixed graphs, generalising work of Achlioptas, Coja-Oghlan [ACO08], and Molloy [Mol12] on random graphs, and justifying the performance of stochastic local search algorithms in this regime. Our central proof relies only on elementary techniques, namely the firstmoment method and a quantitative induction, yet it strengthens list-colouring results due to Vu [Vu02], and more recently Davies, Kang, P., and Sereni [DKPS20], and generalises state-of-the-art bounds from Ramsey theory in the context of sparse graphs. It further yields an approximately tight lower bound on the number of colourings, also known as the partition function of the Potts model, with implications for efficient approximate counting. The research leading to these results was partially supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)-428212407 (E. Hurley).

The List Chromatic Number Of Graphs With Small Clique Number

Abstract: We prove that every triangle-free graph with maximum degree Δ has list chromatic number at most ( 1 + o ( 1 ) ) Δ ln  Δ . This matches the best-known upper bound for graphs of girth at least 5. We also provide a new proof that for any r ≥ 4 every K r -free graph has list-chromatic number at most 200 r Δ ln  ln  Δ ln  Δ .


During training the adversarial examples are constructed with = 0.0157 (4/255) as suggested by #REFR .

Next, we evaluated our approach on the CIFAR-10 dataset using FGSM adversarial training.

We trained the CIFAR-10 model using Wide-Resnet 32-10 model and standard hyperparameters used by Madry et al. #OTHEREFR .

Same value is used to measure robust accuracy during test time. The results are shown in Table I .

Conventional adversarial training on the CIFAR-10 dataset took around 50 hours to converge.

By using our approach, we were able to decrease the training time by a factor of 1.98 and bring it down to 25 hours.

The robust accuracy of the model decreased marginally by 0.64%.

This shows that we were able to achieve a similar level of robustness by using a subset of training data.

Do We Need Entire Training Data For Adversarial Training?

Abstract: Deep Neural Networks (DNNs) are being used to solve a wide range of problems in many domains including safetycritical domains like self-driving cars and medical imagery. DNNs suffer from vulnerability against adversarial attacks. In the past few years, numerous approaches have been proposed to tackle this problem by training networks using adversarial training. Almost all the approaches generate adversarial examples for the entire training dataset, thus increasing the training time drastically. We show that we can decrease the training time for any adversarial training algorithm by using only a subset of training data for adversarial training. To select the subset, we filter the adversarially-prone samples from the training data. We perform a simple adversarial attack on all training examples to filter this subset. In this attack, we add a small perturbation to each pixel and a few grid lines to the input image. We perform adversarial training on the adversarially-prone subset and mix it with vanilla training performed on the entire dataset. Our results show that when our method-agnostic approach is plugged into FGSM [9], we achieve a speedup of 3.52x on MNIST and 1.98x on the CIFAR-10 dataset with comparable robust accuracy. We also test our approach on state-of-the-art Free adversarial training [24] and achieve a speedup of 1.2x in training time with a marginal drop in robust accuracy on the ImageNet dataset.

Towards Deep Learning Models Resistant To Adversarial Attacks

Abstract: Recent work has demonstrated that neural networks are vulnerable to adversarial examples, i.e., inputs that are almost indistinguishable from natural data and yet classified incorrectly by the network. In fact, some of the latest findings suggest that the existence of adversarial attacks may be an inherent weakness of deep learning models. To address this problem, we study the adversarial robustness of neural networks through the lens of robust optimization. This approach provides us with a broad and unifying view on much of the prior work on this topic. Its principled nature also enables us to identify methods for both training and attacking neural networks that are reliable and, in a certain sense, universal. In particular, they specify a concrete security guarantee that would protect against any adversary. These methods let us train networks with significantly improved resistance to a wide range of adversarial attacks. They also suggest the notion of security against a first-order adversary as a natural and broad security guarantee. We believe that robustness against such well-defined classes of adversaries is an important stepping stone towards fully resistant deep learning models.


In #REFR , a method to improve feature matching with depth estimation has been introduced, opening a potential to enhance place recognition with estimated depth.

This approach has exhibited superior robustness to changes in appearance.

To efficiently translate place-distinctive image encoding into localization descriptors, the negative and positive samples were trained in pairs using weak supervision in NetVLAD #OTHEREFR with a triplet loss #OTHEREFR . Radenovic et al.

discovered more consistent and distinctive feature representation from images, such as contrastive loss with generalized mean (GeM) pooling #OTHEREFR .

The study considers every image with sufficiently large number of co-observed 3D points or similar features for a training pair.

This differs from weak supervision with triplet loss, which selects pairs by the image location and their descriptor distance.

In recent studies, methods unifying local and global features have been proposed for expansion over large-scale visual place recognition #OTHEREFR to geometrically verify that the local feature matches after global image searching.

(Lc) 2 : Lidar-Camera Loop Constraints For Cross-Modal Place Recognition

Abstract: Localization has been a challenging task for autonomous navigation. A loop detection algorithm must overcome environmental changes for the place recognition and relocalization of robots. Therefore, deep learning has been extensively studied for the consistent transformation of measurements into localization descriptors. Street view images are easily accessible; however, images are vulnerable to appearance changes. LiDAR can robustly provide precise structural information. However, constructing a point cloud database is expensive, and point clouds exist only in limited places. Different from previous works that train networks to produce shared embedding directly between the 2D image and 3D point cloud, we transform both data into 2.5D depth images for matching. In this work, we propose a novel cross-matching method, called (LC) 2 , for achieving LiDAR localization without a prior point cloud map. To this end, LiDAR measurements are expressed in the form of range images before matching them to reduce the modality discrepancy. Subsequently, the network is trained to extract localization descriptors from disparity and range images. Next, the best matches are employed as a loop factor in a pose graph. Using public datasets that include multiple sessions in significantly different lighting conditions, we demonstrated that LiDAR-based navigation systems could be optimized from image databases and vice versa.

Look No Deeper: Recognizing Places From Opposing Viewpoints Under Varying Scene Appearance Using Single-View Depth Estimation

Abstract: Visual place recognition (VPR) - the act of recognizing a familiar visual place - becomes difficult when there is extreme environmental appearance change or viewpoint change. Particularly challenging is the scenario where both phenomena occur simultaneously, such as when returning for the first time along a road at night that was previously traversed during the day in the opposite direction. While such problems can be solved with panoramic sensors, humans solve this problem regularly with limited field-of-view vision and without needing to constantly turn around. In this paper, we present a new depth- and temporal-aware visual place recognition system that solves the opposing viewpoint, extreme appearance-change visual place recognition problem. Our system performs sequence-to-single frame matching by extracting depth-filtered keypoints using a state-of-the-art depth estimation pipeline, constructing a keypoint sequence over multiple frames from the reference dataset, and comparing these keypoints to the keypoints extracted from a single query image. We evaluate the system on a challenging benchmark dataset and show that it consistently outperforms state-of-the-art techniques. We also develop a range of diagnostic simulation experiments that characterize the contribution of depth-filtered keypoint sequences with respect to key domain parameters including the degree of appearance change and camera motion.


Table I: Code Context Extraction Comparison

The well-established 'Easy to Find' #REFR features such as 'TODO', 'FIXME', and 'XXX' were excluded in the manual validation.

Context in prior work #OTHEREFR Code Context in SoCCMiner #OTHEREFR public #OTHEREFR with 'Sense2Vec' #OTHEREFR .

Sense2Vec captures contextually similar words with its deep word embedding.

For example, for feature 'problematic', Sense2Vec returns multiple semantically similar words such as 'troublesome', 'undesirable', 'contentious', 'separate issue', etc., We use these extrapolated features (words) to evaluate our code comments for SATD.

Sense2Vec extrapolated 'Easy to Find' features such as ('sabotaging', 'damaging', 'counter-productive', etc.,) are passed to the first round of automated SATD annotation.

The first author manually evaluates the results by reading the comments.

During the validation process, the first author creates the heuristics for 'Hard to Find' #OTHEREFR This iterative process involved multiple iterations of annotation, manual evaluation, creating heuristics (pattern matching), and creating features for the 'Hard to Find' SATD.

SATD Annotation: Features that attract too many false positives are removed from the SATD feature list.

For example, 'cause problem' will capture the comment '// Empty bboxes can cause problems' which sound right at first.

Later, after manual scanning, several header comments had the following comment content '//This won't cause problems'.

Such features are removed from the SATD feature list to avoid overfitting features.

Pentacet Data -23 Million Contextual Code Comments And 500,000 Satd Comments

Abstract: Most Self-Admitted Technical Debt (SATD) research utilizes explicit SATD features such as 'TODO' and 'FIXME' for SATD detection. A closer look reveals several SATD research uses simple SATD ('Easy to Find') code comments without the contextual data (preceding and succeeding source code context). This work addresses this gap through PENTACET (or 5C dataset) data. PENTACET is a large Curated Contextual Code Comments per Contributor and the most extensive SATD data. We mine 9,096 Open Source Software Java projects with a total of 435 million LOC. The outcome is a dataset with 23 million code comments, preceding and succeeding source code context for each comment, and more than 500,000 comments labeled as SATD, including both 'Easy to Find' and 'Hard to Find' SATD. We believe PENTACET data will further SATD research using Artificial Intelligence techniques.

Identifying Self-Admitted Technical Debts With Jitterbug: A Two-Step Approach

Abstract: Keeping track of and managing the self-admitted technical debts (SATDs) is important to maintaining a healthy software project. This requires much time and effort from human experts to identify these SATDs manually. Currently, automated solutions do not have high enough precision and recall in identifying SATDs to fully automate the process. To solve the above problems, we propose a two-step framework called Jitterbug for identifying SATDs by first finding the "easy to find" SATDs automatically with close to 100% precision via a novel pattern recognition technique, then applying machine learning techniques to assist human experts in manually identifying the rest "hard to find" SATDs with reduced human effort. Our simulation studies on ten software projects show that Jitterbug can identify SATDs more efficiently (with less human effort) than the prior state of the art methods.


The output of the embedding layer is fed into a convolution layer with kernels of any size. Several convolution layers can be stacked #REFR .

Various kernels can be used to extract various features from the input data points.

In general, kernel filters are designed to be smaller than the input to allow for the sharing of kernel weights across input dimensions.

In file fragment classification, the input to the CNN is either a vector of 4096 or 512 dimensions, equivalent to fragments of 4KB or 512 bytes, respectively.

The input is then transmitted to an embedding layer that converts the input vector into a continuous (4096,32) or (512,32) dimensional vector, dependent on the fragment size.

The embedding layer is a preliminary step to prepare the input vectors for subseque CNN layers.

As a result of stacking standard convolution layers when developing a Deep Neural Network (DNN), in which the model can have a large number of parameters.

File Fragment Classification Using Light-Weight Convolutional Neural Networks

Abstract: In digital forensics, file fragment classification is an important step toward completing file carving process. There exist several techniques to identify the type of file fragments without relying on meta-data, such as using features like header/footer and N-gram to identify the fragment type. Recently, convolutional neural network (CNN) models have been used to build classification models to achieve this task. However, the number of parameters in CNNs tends to grow exponentially as the number of layers increases. This results in a dramatic increase in training and inference time. In this paper, we propose lightweight file fragment classification models based on depthwise separable CNNs. The evaluation results show that our proposed models provide faster inference time with comparable accuracy as compared to the state-of-art CNN based models. In particular, our models were able to achieve an accuracy of 79% on the FFT-75 dataset with nearly 100K parameters and 164M FLOPs, which is 4x smaller and 6x faster than the state-of-the-art classifier in the literature.

Fifty: Large-Scale File Fragment Type Identification Using Neural Networks

Abstract: We present FiFTy, a modern file type identification tool for memory forensics and data carving. In contrast to previous approaches based on hand-crafted features, we design a compact neural network architecture, which uses a trainable embedding space, akin to successful natural language processing models. Our approach dispenses with explicit feature extraction which is a bottleneck in legacy systems. We evaluate the proposed method on a novel dataset with 75 file types - the most diverse and balanced dataset reported to date. FiFTy consistently outperforms all baselines in terms of speed, accuracy and individual misclassification rates. We achieved an average accuracy of 77.5% with processing speed of approx 38 sec/GB, which is better and more than an order of magnitude faster than the previous state-of-the-art tool - Sceadan (69% at 9 min/GB). Our tool and the corresponding dataset are available publicly online.

Several convolution layers
embedding layer


Compared to the stability result in #REFR (see Theorem 2.5), in our result we observe a square-root dependence of the reconstruction error bound on the noise level τ for small noise levels.

To see how this compares to the the existing dimension-dependent recovery guarantee (2.6) in [ARR14; LS17; JKS17] we first reformulate (3.2) as

Ignoring logarithmic factors, for noise levels

this significantly improves over the dimension-dependent recovery guarantee (2.6).

In contrast, the reconstruction error bound in Theorem 2.5 becomes constant whenever the noise level τ is smaller than a certain threshold.

The bound (3.1) in Theorem 3.1 becomes worse when the noise level τ becomes smaller.

This reflects the instability result in #OTHEREFR , see Theorem 2.4, which shows the existence of an alternative solution which amplifies the output error by a dimension factor.

How Robust Is Randomized Blind Deconvolution Via Nuclear Norm Minimization Against Adversarial Noise?

Abstract: In this paper, we study the problem of recovering two unknown signals from their convolution, which is commonly referred to as blind deconvolution. Reformulation of blind deconvolution as a low-rank recovery problem has led to multiple theoretical recovery guarantees in the past decade due to the success of the nuclear norm minimization heuristic. In particular, in the absence of noise, exact recovery has been established for sufficiently incoherent signals contained in lower-dimensional subspaces. However, if the convolution is corrupted by additive bounded noise, the stability of the recovery problem remains much less understood. In particular, existing reconstruction bounds involve large dimension factors and therefore fail to explain the empirical evidence for dimension-independent robustness of nuclear norm minimization. Recently, theoretical evidence has emerged for ill-posed behavior of low-rank matrix recovery for sufficiently small noise levels. In this work, we develop improved recovery guarantees for blind deconvolution with adversarial noise which exhibit square-root scaling in the noise level. Hence, our results are consistent with existing counterexamples which speak against linear scaling in the noise level as demonstrated for related low-rank matrix recovery problems.

On The Convex Geometry Of Blind Deconvolution And Matrix Completion

Abstract: Low-rank matrix recovery from structured measurements has been a topic of intense study in the last decade and many important problems like matrix completion and blind deconvolution have been formulated in this framework. An important benchmark method to solve these problems is to minimize the nuclear norm, a convex proxy for the rank. A common approach to establish recovery guarantees for this convex program relies on the construction of a so-called approximate dual certificate. However, this approach provides only limited insight in various respects. Most prominently, the noise bounds exhibit seemingly suboptimal dimension factors. In this paper we take a novel, more geometric viewpoint to analyze both the matrix completion and the blind deconvolution scenario. We find that for both these applications the dimension factors in the noise bounds are not an artifact of the proof, but the problems are intrinsically badly conditioned. We show, however, that bad conditioning only arises for very small noise levels: Under mild assumptions that include many realistic noise levels we derive near-optimal error estimates for blind deconvolution under adversarial noise.