Another advantage is transfer learning i.e., training of low-resource languages in combination with high resource languages improves the translation quality of low-resource languages #REFR .

Neural machine translation (NMT) has established a number of cutting-edge benchmarks in machine translation tasks #OTHEREFR , #OTHEREFR .

It is an encoder-decoder framework based on a sequence-to-sequence prediction model where the encoder generates the context vector by taking a source-side word as input, and the decoder decodes this generated context vector into target sequences.

Recent works ( [3] - #OTHEREFR ) have extended the NMT approach to support multilingual translation, i.e., training a single model that can translate between multiple languages.

There are several reasons for shifting researchers' interest from bilingual to multilingual machine translation.

First, single model training for large number of languages makes multilingual model more cost-effective than multiple bilingual models.

One of the fine examples of transfer learning-based NMT under multilingual scenario is Zero-Shot Translation (ZST) #OTHEREFR .

ZST is an unsupervised approach in which pretrained translation models are tested on related unseen language pairs.

There has been a significant amount of works done on multilingual machine translation, with the majority of the works focusing on translation between language pairs with English as a prominent language #OTHEREFR - #OTHEREFR .

However, very few works on non-English language pairs exist #OTHEREFR , #OTHEREFR .

One of the reasons for less multilingual systems on non-English language pairs is insufficient training data for such language pairs.

Exploiting Multilingualism In Low-Resource Neural Machine Translation Via Adversarial Learning

Abstract: Generative Adversarial Networks (GAN) offer a promising approach for Neural Machine Translation (NMT). However, feeding multiple morphologically languages into a single model during training reduces the NMT's performance. In GAN, similar to bilingual models, multilingual NMT only considers one reference translation for each sentence during model training. This single reference translation limits the GAN model from learning sufficient information about the source sentence representation. Thus, in this article, we propose Denoising Adversarial Auto-encoderbased Sentence Interpolation (DAASI) approach to perform sentence interpolation by learning the intermediate latent representation of the source and target sentences of multilingual language pairs. Apart from latent representation, we also use the Wasserstein-GAN approach for the multilingual NMT model by incorporating the model generated sentences of multiple languages for reward computation. This computed reward optimizes the performance of the GAN-based multilingual model in an effective manner. We demonstrate the experiments on low-resource language pairs and find that our approach outperforms the existing state-of-the-art approaches for multilingual NMT with a performance gain of up to 4 BLEU points. Moreover, we use our trained model on zero-shot language pairs under an unsupervised scenario and show the robustness of the proposed approach.

Transfer Learning For Low-Resource Neural Machine Translation

Abstract: The encoder-decoder framework for neural machine translation (NMT) has been shown effective in large data scenarios, but is much less effective for low-resource languages. We present a transfer learning method that significantly improves Bleu scores across a range of low-resource languages. Our key idea is to first train a high-resource language pair (the parent model), then transfer some of the learned parameters to the low-resource pair (the child model) to initialize and constrain training. Using our transfer learning method we improve baseline NMT models by an average of 5.6 Bleu on four low-resource language pairs. Ensembling and unknown word replacement add another 2 Bleu which brings the NMT performance on low-resource machine translation close to a strong syntax based machine translation (SBMT) system, exceeding its performance on one language pair. Additionally, using the transfer learning model for re-scoring, we can improve the SBMT system by an average of 1.3 Bleu, improving the state-of-the-art on low-resource machine translation.


Table 1 reports the test error at the 200 epoch mark, replicating the results reported in Table 2 in #REFR .

We replicate here the experiment of Rice et al. (2020) exhibiting robust overfitting for AT training. As in Rice et al.

(2020) we use preactivation ResNet18 on CIFAR-10 with PGD attacks of 10 steps, and with SGD learning rate decay at epochs 100 and 150. Further experimental setup is discussed in Appendix E.1. We train AT with PGD of #OTHEREFR , TRADES of #OTHEREFR

(2019) , and HR with r ≥ 0, > 0 and α = 0.

For all algorithms, we use the same PGD attack on training with = 8/255 and 10 attack steps.

Figure 3 shows the test loss and error up to 200 training epochs when using pre-activation ResNet18 under ∞ attacks.

AT clearly exhibits robust overfitting as the test loss drastically increases with more training epochs.

The test error also increases but perhaps not as drastically.

This indicates that although the number of errors made eventually settles, worryingly, the errors are made with increasing confidence as quantified by the crossentropy loss. Similar to Rice et al.

(2020), we find that TRADES #OTHEREFR without early stopping also experiences robust overfitting although to a lesser extent than regular PGD training.

HR training with modest parameter values r effectively mitigates this phenomenon and results in a monotonically decreasing test loss as a function of training epochs eliminating the need for early stopping.

Certified Robust Neural Networks: Generalization And Corruption Resistance

Abstract: Adversarial training aims to reduce the problematic susceptibility of modern neural networks to small data perturbations. Surprisingly, overfitting is a major concern in adversarial training of neural networks despite being mostly absent in standard training. We provide here theoretical evidence for this peculiar "robust overfitting" phenomenon. Subsequently, we advance a novel loss function which we show both theoretically as well as empirically to enjoy a certified level of robustness against data evasion and poisoning attacks while ensuring guaranteed generalization. We indicate through careful numerical experiments that our resulting holistic robust (HR) training procedure yields state of the art performance in terms of adversarial error loss. Finally, we indicate that HR training can be interpreted as a direct extension of adversarial training and comes with a negligible additional computational burden. A readyto-use python library implementing our algorithm is available at https://github.com/ RyanLucas3/HR_Neural_Networks.

Overfitting In Adversarially Robust Deep Learning

Abstract: It is common practice in deep learning to use overparameterized networks and train for as long as possible; there are numerous studies that show, both theoretically and empirically, that such practices surprisingly do not unduly harm the generalization performance of the classifier. In this paper, we empirically study this phenomenon in the setting of adversarially trained deep networks, which are trained to minimize the loss under worst-case adversarial perturbations. We find that overfitting to the training set does in fact harm robust performance to a very large degree in adversarially robust training across multiple datasets (SVHN, CIFAR-10, CIFAR-100, and ImageNet) and perturbation models ($\ell_\infty$ and $\ell_2$). Based upon this observed effect, we show that the performance gains of virtually all recent algorithmic improvements upon adversarial training can be matched by simply using early stopping. We also show that effects such as the double descent curve do still occur in adversarially trained models, yet fail to explain the observed overfitting. Finally, we study several classical and modern deep learning remedies for overfitting, including regularization and data augmentation, and find that no approach in isolation improves significantly upon the gains achieved by early stopping. All code for reproducing the experiments as well as pretrained model weights and training logs can be found at this https URL.


Since robustness #REFR is another important metric that evaluates the generalization ability to handle unseen data of the trained model, we obtain multiple trained code models and study their adversarial robustness in this RQ.

In order to assess the effectiveness of the data augmentation methods in Section 3 in source code learning, we design three research questions, as follows:

• RQ1: Can existing data augmentation methods produce accurate code models? Accuracy is the basic metric to evaluate the performance of a trained model, and therefore, we first assess whether data augmentation methods from NLP and graph learning (as introduced in Section 3) are able to improve the accuracy of the code models, based on the comparisons with two baseline approaches, namely, the training approach that does not use any data augmentation method, and the training approach that only uses the simple code refactoring method (as introduced in Section 2.2).

In this RQ, first, we prepare the original training data and randomly initialize code models, and then train these models using different data augmentation methods as listed in Fig. 2 .

Specifically, we compare these methods in terms of two metrics, namely, 1) the convergence speed of the model and 2) the final accuracy of the model with fixed training epochs.

• RQ2: Can existing data augmentation methods produce robust code models?

According to the literature #OTHEREFR , data augmentation is helpful for improving the adversarial robustness of DNN models in other fields (e.g., computer vision).

Thus, we also explore whether this conclusion also holds for code models.

Concretely, we employ two state-of-the-art adversarial attacks, namely, MHM #OTHEREFR and ALERT #OTHEREFR , to evaluate the robustness improvement of the code models that are trained using data augmentation.

• RQ3: How does data volume affect the effectiveness of data augmentation methods? Finally, since the initial purpose of data augmentation is to solve the problem of the lack of labeled data, it is necessary to further investigate the effectiveness of data augmentation methods in a more practical scenario, namely, the case when there is no sufficient training data.

To do so, we reduce the size of the training set of each dataset and repeat the experiments in RQ1 and RQ2, and we then check whether data augmentation methods are still useful.

Boosting Source Code Learning With Data Augmentation: An Empirical Study

Abstract: The next era of program understanding is being propelled by the use of machine learning to solve software problems. Recent studies have shown surprising results of source code learning, which applies deep neural networks (DNNs) to various critical software tasks, e.g., bug detection and clone detection. This success can be greatly attributed to the utilization of massive high-quality training data, and in practice, data augmentation, which is a technique used to produce additional training data, has been widely adopted in various domains, such as computer vision. However, in source code learning, data augmentation has not been extensively studied, and existing practice is limited to simple syntax-preserved methods, such as code refactoring. Essentially, source code is often represented in two ways, namely, sequentially as text data and structurally as graph data, when it is used as training data in source code learning. Inspired by these analogy relations, we take an early step to investigate whether data augmentation methods that are originally used for text and graphs are effective in improving the training quality of source code learning. To that end, we first collect and categorize data augmentation methods in the literature. Second, we conduct a comprehensive empirical study on four critical tasks and 11 DNN architectures to explore the effectiveness of 12 data augmentation methods (including code refactoring and 11 other methods for text and graph data). Our results identify the data augmentation methods that can produce more accurate and robust models for source code learning, including those based on mixup (e.g., SenMixup for texts and Manifold-Mixup for graphs), and those that slightly break the syntax of source code (e.g., random swap and random deletion for texts).

Robustness And Generalization

Abstract: We derive generalization bounds for learning algorithms based on their robustness: the property that if a testing sample is"similar"to a training sample, then the testing error is close to the training error. This provides a novel approach, different from the complexity or stability arguments, to study generalization of learning algorithms. We further show that a weak notion of robustness is both sufficient and necessary for generalizability, which implies that robustness is a fundamental property for learning algorithms to work.

multiple trained code
adversarial robustness


For example, while Spanish hashtags are more entertainment-oriented, Catalan hashtags tend to be more political; this is consistent with prior work that shows Catalan is used online to mark political identity #REFR .

Effects are greatest when C y is 2-3 hours ahead (i.e. much further east) of C x .

Figure S2 : Fraction of unique hashtags associated with sample of languages for each of four macro-topics, sorted left to right from largest to smallest proportion of political hashtags.

Finnish and German hashtags are largely political, while Russian, Spanish, and Hungarian hashtags tend to be more entertainment-oriented.

Portuguese hashtags are much more focused on entertainment and sports compared to politics.

This figure also illustrates how language choice can be a meaningful sociolinguistic variable.

Table S5 : Summary of locus-specific causal estimates for each topic macro-category.

Out of the 464 possible loci from 232 MCPs, ATT estimates were only calculated for a subset of loci with sufficient data.

The remaining columns show the percent of estimates where multilinguals have a significantly positive, negative, and no significant effect (based on heteroskedasticityconsistent standard error estimation with p < 0.05.

Figure S3 : Distributions of tweets and hashtags assigned to all 50 topics with multilingual contextualized topic modeling.

The top plot shows the fraction of all tweets assigned to each topic.

Bridging Nations: Quantifying The Role Of Multilinguals In Communication On Social Media

Abstract: Social media enables the rapid spread of many kinds of information, from memes to social movements. However, little is known about how information crosses linguistic boundaries. We apply causal inference techniques on the European Twitter network to quantify multilingual users' structural role and communication influence in cross-lingual information exchange. Overall, multilinguals play an essential role; posting in multiple languages increases betweenness centrality by 13%, and having a multilingual network neighbor increases monolinguals' odds of sharing domains and hashtags from another language 16-fold and 4-fold, respectively. We further show that multilinguals have a greater impact on diffusing information less accessible to their monolingual compatriots, such as information from faraway countries and content about regional politics, nascent social movements, and job opportunities. By highlighting information exchange across borders, this work sheds light on a crucial component of how information and ideas spread around the world.

S\'I O No, Qu\`E Penses? Catalonian Independence And Linguistic Identity On Social Media

Abstract: Political identity is often manifested in language variation, but the relationship between the two is still relatively unexplored from a quantitative perspective. This study examines the use of Catalan, a language local to the semi-autonomous region of Catalonia in Spain, on Twitter in discourse related to the 2017 independence referendum. We corroborate prior findings that pro-independence tweets are more likely to include the local language than anti-independence tweets. We also find that Catalan is used more often in referendum-related discourse than in other contexts, contrary to prior findings on language variation. This suggests a strong role for the Catalan language in the expression of Catalonian political identity.


We validate the performance of our algorithm on the Im-ageNet dataset #REFR , which consists of 1.2M training images and 50, 000 validation images.

We take 16 batches of data to perform the reconstruction of PTQ.

We fix the batch size to 256 for ResNet-18 and MobileNetV2, and 128 for ResNet-50.

In addition, our experimental analysis of the expanded batch size is shown in Sec. 4.3. Our data preprocessing follows #OTHEREFR as everyone does.

We use the Pytorch library #OTHEREFR to complete our algorithm.

We perform our experiments on an Nvidia Tesla A100 as well as an Intel(R) Xeon(R) Platinum 8336C CPU.

Solving Oscillation Problem In Post-Training Quantization Through A Theoretical Perspective

Abstract: Post-training quantization (PTQ) is widely regarded as one of the most efficient compression methods practically, benefitting from its data privacy and low computation costs. We argue that an overlooked problem of oscillation is in the PTQ methods. In this paper, we take the initiative to explore and present a theoretical proof to explain why such a problem is essential in PTQ. And then, we try to solve this problem by introducing a principled and generalized framework theoretically. In particular, we first formulate the oscillation in PTQ and prove the problem is caused by the difference in module capacity. To this end, we define the module capacity (ModCap) under data-dependent and data-free scenarios, where the differentials between adjacent modules are used to measure the degree of oscillation. The problem is then solved by selecting top-k differentials, in which the corresponding modules are jointly optimized and quantized. Extensive experiments demonstrate that our method successfully reduces the performance drop and is generalized to different neural networks and PTQ methods. For example, with 2/4 bit ResNet-50 quantization, our method surpasses the previous state-of-the-art method by 1.9%. It becomes more significant on small model quantization, e.g. surpasses BRECQ method by 6.61% on MobileNetV2 ×0.5.

Imagenet Large Scale Visual Recognition Challenge

Abstract: The ImageNet Large Scale Visual Recognition Challenge is a benchmark in object category classification and detection on hundreds of object categories and millions of images. The challenge has been run annually from 2010 to present, attracting participation from more than fifty institutions. This paper describes the creation of this benchmark dataset and the advances in object recognition that have been possible as a result. We discuss the challenges of collecting large-scale ground truth annotation, highlight key breakthroughs in categorical object recognition, provide a detailed analysis of the current state of the field of large-scale image classification and object detection, and compare the state-of-the-art computer vision accuracy with human accuracy. We conclude with lessons learned in the five years of the challenge, and propose future directions and improvements.


To illustrate the difficulty of training a UniKW-AT model, we also ran baseline experiments using a common TC-ResNet8 #REFR model.

The core results of our work on the GSCV1 and Audioset can be seen in Table 2 .

The results show that even though TC-ResNet8 can achieve excellent performance on GSCV1 (96.72 Acc), it fails to provide meaningful performance on AT (8.67 mAP).

Note that TC-ResNet8's performance on GSCV1 improves against the publicly reported result (96.10 → 96.72) due to UniKW-AT training, where 60h of "noise" samples from Audioset enhance the model's robustness to false alarms.

As we can see, our proposed UiT-XS achieves competitive results compared to the previous Mo-bileNetV2 (MBv2) based method for UniKW-AT, as well as other works in the literature regarding KWS and AT performance.

Unified Keyword Spotting And Audio Tagging On Mobile Devices With Transformers

Abstract: Keyword spotting (KWS) is a core human-machine-interaction front-end task for most modern intelligent assistants. Recently, a unified (UniKW-AT) framework has been proposed that adds additional capabilities in the form of audio tagging (AT) to a KWS model. However, previous work did not consider the real-world deployment of a UniKW-AT model, where factors such as model size and inference speed are more important than performance alone. This work introduces three mobile-device deployable models named Unified Transformers (UiT). Our best model achieves an mAP of 34.09 on Audioset, and an accuracy of 97.76 on the public Google Speech Commands V1 dataset. Further, we benchmark our proposed approaches on four mobile platforms, revealing that the proposed UiT models can achieve a speedup of 2-6 times against a competitive MobileNetV2.

Temporal Convolution For Real-Time Keyword Spotting On Mobile Devices

Abstract: Keyword spotting (KWS) plays a critical role in enabling speech-based user interactions on smart devices. Recent developments in the field of deep learning have led to wide adoption of convolutional neural networks (CNNs) in KWS systems due to their exceptional accuracy and robustness. The main challenge faced by KWS systems is the trade-off between high accuracy and low latency. Unfortunately, there has been little quantitative analysis of the actual latency of KWS models on mobile devices. This is especially concerning since conventional convolution-based KWS approaches are known to require a large number of operations to attain an adequate level of performance. In this paper, we propose a temporal convolution for real-time KWS on mobile devices. Unlike most of the 2D convolution-based KWS approaches that require a deep architecture to fully capture both low- and high-frequency domains, we exploit temporal convolutions with a compact ResNet architecture. In Google Speech Command Dataset, we achieve more than \textbf{385x} speedup on Google Pixel 1 and surpass the accuracy compared to the state-of-the-art model. In addition, we release the implementation of the proposed and the baseline models including an end-to-end pipeline for training models and evaluating them on mobile devices.


Scene Graph Generation: There are mainly two mainstream methods for scene graph generation: Based on context modeling or graph convolutional network (GCN) #REFR .

The first approach is focused on modeling global information and widely adapted message passing between all entities features #OTHEREFR .

A number of them #OTHEREFR harnessed sequential models e.g., LSTM #OTHEREFR , GRU #OTHEREFR and Transformer #OTHEREFR . Chen et al.

#OTHEREFR use two stacks of Transformer to encode global information. Zellers et al.

#OTHEREFR uses LSTM to encode global context that informs relation prediction.

However, merely modeling global context is not sufficient for Scene graph tasks.

Landmark: Language-Guided Representation Enhancement Framework For Scene Graph Generation

Abstract: Scene graph generation (SGG) is a sophisticated task that suffers from both complex visual features and dataset long-tail problem. Recently, various unbiased strategies have been proposed by designing novel loss functions and data balancing strategies. Unfortunately, these unbiased methods fail to emphasize language priors in feature refinement perspective. Inspired by the fact that predicates are highly correlated with semantics hidden in subject-object pair and global context, we propose LANDMARK (LANguage-guiDed representation enhanceMent frAmewoRK) that learns predicate-relevant representations from language-vision interactive patterns, global language context and pair-predicate correlation. Specifically, we first project object labels to three distinctive semantic embeddings for different representation learning. Then, Language Attention Module (LAM) and Experience Estimation Module (EEM) process subject-object word embeddings to attention vector and predicate distribution, respectively. Language Context Module (LCM) encodes global context from each word embedding, which avoids isolated learning from local information. Finally, modules outputs are used to update visual representations and SGG model's prediction. All language representations are purely generated 1

Semi-Supervised Classification With Graph Convolutional Networks

Abstract: We present a scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs. We motivate the choice of our convolutional architecture via a localized first-order approximation of spectral graph convolutions. Our model scales linearly in the number of graph edges and learns hidden layer representations that encode both local graph structure and features of nodes. In a number of experiments on citation networks and on a knowledge graph dataset we demonstrate that our approach outperforms related methods by a significant margin.

scene graph generation
graph convolutional network


V-COCO is a subset of MS-COCO #REFR with HOI annotations, including 10,346 images (2,533 for training, 2,867 for validation and 4,946 for testing).

HICO-DET [3] and V-COCO #OTHEREFR are two widely-used HOI benchmarks.

HICO-DET contains 47,776 images, with 38,118 for training and 9,658 for testing.

There are 600 HOI categories in HICO-DET, consisting of 117 interaction classes and 80 object classes.

Each HOI category is composed of an interaction and an object.

It has 80 object categories same with HICO-DET and 29 interaction categories. Evaluation metrics.

For HICO-DET, we adopt the commonly used mAP metric [3] . Each prediction is a 〈human, interaction, object〉 triplet.

A prediction is a true positive only when the human and object bounding boxes both have IoU >0.5 w.r.t. ground truth and the interaction classification result is correct.

We evaluate the performance in two different settings following [3] .

In the known object setting, for each HOI category, we evaluate the prediction only on the images containing the target object category.

Category Query Learning For Human-Object Interaction Classification

Abstract: Unlike most previous HOI methods that focus on learning better human-object features, we propose a novel and complementary approach called category query learning. Such queries are explicitly associated to interaction categories, converted to image specific category representation via a transformer decoder, and learnt via an auxiliary image-level classification task. This idea is motivated by an earlier multi-label image classification method, but is for the first time applied for the challenging humanobject interaction classification task. Our method is simple, general and effective. It is validated on three representative HOI baselines and achieves new state-of-theart results on two benchmarks. Code will be available at https://github.com/charles-xie/CQL.

Microsoft Coco: Common Objects In Context

Abstract: We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding. This is achieved by gathering images of complex everyday scenes containing common objects in their natural context. Objects are labeled using per-instance segmentations to aid in precise object localization. Our dataset contains photos of 91 objects types that would be easily recognizable by a 4 year old. With a total of 2.5 million labeled instances in 328k images, the creation of our dataset drew upon extensive crowd worker involvement via novel user interfaces for category detection, instance spotting and instance segmentation. We present a detailed statistical analysis of the dataset in comparison to PASCAL, ImageNet, and SUN. Finally, we provide baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model.


For feature extraction, after examine the performance of several backbones, we select the ResNet152 #REFR as the backbone for Mask R-CNN and ResNet101 for DeepLabv3.

In the training session of our baseline method for panoptic segmentation, we trained a single model for instance segmentation using Mask R-CNN #OTHEREFR , and a single model for semantic segmentation using DeepLabv3 #OTHEREFR architecture.

No ensemble, test time tricks or post processing tricks at any level used in this baseline.

Note that for all experiments including the baseline, the optimizer is consistent. We use Stochastic Gradient Descent(SGD) with momentum.

For instance segmentation, a multi-step learning scheduler (LR) with initial warm-up is adopted, whereas for semantic segmentation, an additional LR is used to generate a decaying LR curve using a multiplicative factor.

Our panoptic segmentation builds upon instance and semantic segmentation therefore the performance of these two has crucial influence on the final panoptic segmentation results.

In the following section, we will report in detail, how we improve this baseline by studying different strategies we used for instance and semantic segmentation respectively.

Joint Coco And Mapillary Workshop At Iccv 2019: Coco Panoptic Segmentation Challenge Track Technical Report: Ensembling Instance And Semantic For Panoptic Segmentation

Abstract: In this technical report, we demonstrate our solution for the 2019 COCO panoptic segmentation task. Our method first performs instance segmentation and semantic segmentation separately, then combines the two to generate panoptic segmentation results. To enhance the performance, we add several expert models of Mask R-CNN in instance segmentation to tackle the data imbalance problem in the training data; also HTC model is adopted yielding our best instance segmentation results. In semantic segmentation, we trained several models with various backbones and use an ensemble strategy which further boosts the segmentation results. In the end, we analyze various combinations of instance and semantic segmentation, and report on their performance for the final panoptic segmentation results. Our best model achieves P Q 47.1 on 2019 COCO panoptic testdev data.

Deep Residual Learning For Image Recognition

Abstract: Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.


The GCN model can update the presentation of the nodes in the graph by itself and their neighbors #REFR .

The RKT model combines past interactions with the attention mechanism to make predictions #OTHEREFR without considering the relationship between students, exercises, and KCs.

A non-linear conducted graph is more complicated than a tree structure.

Information with a graph structure can describe connections and entities in the real world in an intuitive manner #OTHEREFR .

A unique type of neural network called a GCN operates directly on data with a graph structure.

Applying numerous graph convolutional layers, the GCN model can make sure that the updated nodes properly represent both the features of higher-order neighbors and the features of neighboring nodes.

There exist some knowledge tracing models focusing on investigating the potential usage of knowledge graph structures to capture more knowledge.

According to the works of #OTHEREFR , each node represents a student; the nodes are fixed, but the edges are dynamic. This creates a dynamic graph that changes over time.

However, this method ignores the heterogeneous interactions between the students-exercises-KCs.

The GIKT model is proposed to treat the exercise-skill relationship graph as a bipartite graph and then applies the embedding propagation to the GCN to integrate the exercise-skill correlation #OTHEREFR without considering student behavior.

Knowledge Relation Rank Enhanced Heterogeneous Learning Interaction Modeling For Neural Graph Forgetting Knowledge Tracing

Abstract: Recently, knowledge tracing models have been applied in educational data mining such as the Self-attention knowledge tracing model(SAKT), which models the relationship between exercises and Knowledge concepts(Kcs). However, relation modeling in traditional Knowledge tracing models only considers the static question-knowledge relationship and knowledge-knowledge relationship and treats these relationships with equal importance. This kind of relation modeling is difficult to avoid the influence of subjective labeling and considers the relationship between exercises and KCs, or KCs and KCs separately. In this work, a novel knowledge tracing model, named Knowledge Relation Rank Enhanced Heterogeneous Learning Interaction Modeling for Neural Graph Forgetting Knowledge Tracing(NGFKT), is proposed to reduce the impact of the subjective labeling by calibrating the skill relation matrix and the Q-matrix and apply the Graph Convolutional Network(GCN) to model the heterogeneous interactions between students, exercises, and skills. Specifically, the skill relation matrix and Q-matrix are generated by the Knowledge Relation Importance Rank Calibration method(KRIRC). Then the calibrated skill relation matrix, Q-matrix, and the heterogeneous interactions are treated as the input of the GCN to generate the exercise embedding and skill embedding. Next, the exercise embedding, skill embedding, item difficulty, and contingency table are incorporated to generate an exercise relation matrix as the inputs of the Position-Relation-Forgetting attention mechanism. Finally, the Position-Relation-Forgetting attention mechanism is applied to make the predictions. Experiments are conducted on the two public educational datasets and results indicate that the NGFKT model outperforms all baseline models in terms of AUC, ACC, and Performance Stability(PS).


Support vector machines are utilized as decision tree generator to incorporate the spatial structure of featured class distribution #REFR .

Some other methods include decision trees based on fuzzy inference systems #OTHEREFR , feature space #OTHEREFR ensemble and support vector ma-chine based decision trees #OTHEREFR .

Weights of random feature for ensemble decision tree assign weight to each individual attribute for better model variation #OTHEREFR .

The interpretability of featured decision trees has been studied recently inorder to better understand how they make decisions #OTHEREFR .

More literature information about decision trees can be studied from #OTHEREFR .

The problem with all of the impurity measures, as mentioned in #OTHEREFR , is that they are different class distribution functions of the hyperplane on both of its sides and disregard the spatial structure of the regions of its class #OTHEREFR and if the labels of the data are changed without any modification in the significant features of hyperplane edges, it has no effect on the impurity metric.

The proximal hyperplanes generated by multisurface proximal support vector machines (MPSVM) are in such a way that each proximal hyperplane is closer to the class samples and far away from another class samples #OTHEREFR .

Inorder to make the node pure, authors #OTHEREFR produced planes of two clustering at non-leaf node level by selecting an angle bisector. MPSVM algorithm is considered in the binary class.

Therefore, the multiclass problem has been split into binary class considering superior class samples in one group and remaining into another class sample.

As the tree grows bigger, the nodes reach towards its purest form and the subsequent nodes receives less samples.

To avoid this issue, NULL space algorithm #OTHEREFR is deployed #OTHEREFR .

Heterogeneous Oblique Double Random Forest

Abstract: The decision tree ensembles use a single data feature at each node for splitting the data. However, splitting in this manner may fail to capture the geometric properties of the data. Thus, oblique decision trees generate the oblique hyperplane for splitting the data at each non-leaf node. Oblique decision trees capture the geometric properties of the data and hence, show better generalization. The performance of the oblique decision trees depends on the way oblique hyperplanes are generate and the data used for the generation of those hyperplanes. Recently, multiple classifiers have been used in a heterogeneous random forest (RaF) classifier, however, it fails

Geometric Decision Tree

Abstract: In this paper, we present a new algorithm for learning oblique decision trees. Most of the current decision tree algorithms rely on impurity measures to assess the goodness of hyperplanes at each node while learning a decision tree in top-down fashion. These impurity measures do not properly capture the geometric structures in the data. Motivated by this, our algorithm uses a strategy for assessing the hyperplanes in such a way that the geometric structure in the data is taken into account. At each node of the decision tree, we find the clustering hyperplanes for both the classes and use their angle bisectors as the split rule at that node. We show through empirical studies that this idea leads to small decision trees and better performance. We also present some analysis to show that the angle bisectors of clustering hyperplanes that we use as the split rules at each node are solutions of an interesting optimization problem and hence argue that this is a principled method of learning a decision tree.


We use #REFR to extract the pseudoalbedo ground truth of the inputs before the start of the training phase. Recently, Ye et al.

Some works #OTHEREFR have built datasets with variation in both attributes, but still have limitations in that their application is limited to outdoor scene collections #OTHEREFR or object-centric scenes #OTHEREFR (see Tab. 1).

Intrinsic image decomposition. As Weiss et al.

#OTHEREFR have proved, multiple images with varying illumination can provide rich guidance to intrinsic decomposition. Li et al.

#OTHEREFR has succeeded in disentangling intrinsic components without supervision, by enforcing multiple images with varying illumination to have identical albedo. Das et al.

#OTHEREFR has shown state-of-the-art performance by leveraging crosscolor-ratio, the illumination-invariant image gradients in an intrinsic decomposition.

#OTHEREFR have suggested a NeRF framework that enables intrinsic decomposition for a purpose of scene editing.

However, did not address the problem of few-shot view synthesis under varying illumination.

Extremenerf: Few-Shot Neural Radiance Fields Under Unconstrained Illumination

Abstract: In this paper, we propose a new challenge that synthesizes a novel view in a more practical environment, where the number of input multi-view images is limited and illumination variations are significant. Despite recent success, neural radiance fields (NeRF) require a massive amount of input multi-view images taken under constrained illuminations. To address the problem, we suggest ExtremeNeRF, which utilizes occlusion-aware multiview albedo consistency, supported by geometric alignment and depth consistency. We extract intrinsic image components that should be illumination-invariant across different views, enabling direct appearance comparison between the input and novel view under unconstrained illumination. We provide extensive experimental results for an evaluation of the task, using the newly built NeRF Extreme benchmark, which is the first in-the-wild novel view synthesis benchmark taken under multiple viewing directions and varying illuminations. The project page is at https://seokyeong94.github.io/ExtremeNeRF/

Learning Intrinsic Image Decomposition From Watching The World

Abstract: Single-view intrinsic image decomposition is a highly ill-posed problem, and so a promising approach is to learn from large amounts of data. However, it is difficult to collect ground truth training data at scale for intrinsic images. In this paper, we explore a different approach to learning intrinsic images: observing image sequences over time depicting the same scene under changing illumination, and learning single-view decompositions that are consistent with these changes. This approach allows us to learn without ground truth decompositions, and to instead exploit information available from multiple images when training. Our trained model can then be applied at test time to single views. We describe a new learning framework based on this idea, including new loss functions that can be efficiently evaluated over entire sequences. While prior learning-based methods achieve good performance on specific benchmarks, we show that our approach generalizes well to several diverse datasets, including MIT intrinsic images, Intrinsic Images in the Wild and Shading Annotations in the Wild.1


The evaluation on attribute accuracies is similar to the automatic results, except that the accuracy of gender slightly decreases as human evaluators don't take neutral content that fits common gender prototypes as correctly reflecting the gender style, like soldier for male and baby-carer for female #REFR .

The CTRL model leveraged the attributes in finetuning, and achieved better control accuracy and BertScore, but it doesn't produce more diverse responses overall.

Both Director and DASC exhibit the best controllability, and DASC produces more diverse and reasonable responses according to Distinct-2 and BertScore.

We then show human judgement results in Table 2.

The inter-annotator agreement for Acc G , Acc E and Acc Q are 0.65, 0.55 and 0.64 in Cohen's κ, which indicates moderate to substantial agreement.

The agreement of Interestingness and Sensibleness is 0.48 and 0.44 in Pearson's r.

The annotators also check questions without a question mark, which explains the slight difference in Acc Q .

Overall, the rankings of controllability still hold according to human evaluation, with DASC performs the best.

Baseline, Rerank and CTRL have slightly better Sensibleness than weighted decoding methods, which agrees with the commonly observed controllability-quality trade-off in previous literature #OTHEREFR .

All controllable generation methods achieved higher Interestingness score than baseline, which supports the benefits of controllable generation.

DASC achieved the best Interestingness given similar attributes accuracy as Director, indicating the effectiveness of attribute semantic space, which can establish better representations of attribute semantics and a more reasonable approach to compose the control attributes in weighted decoding.

Semantic Space Grounded Weighted Decoding For Multi-Attribute Controllable Dialogue Generation

Abstract: Controlling chatbot utterance generation with multiple attributes such as personalities, emotions and dialogue acts is a practically useful but under-studied problem. We propose a novel controllable generation framework called DASC that possesses strong controllability with weighted decoding paradigm, while improving generation quality with the grounding in an attribute semantics space. Generation with multiple attributes is then intuitively implemented with an interpolation of multiple attribute embeddings. Experiments show that DASC can achieve state-of-the-art control accuracy in 3-aspect controllable generation tasks while also producing interesting and reasonably sensible responses, even if in an outof-distribution robustness test. Visualization of the meaningful representations learned in the attribute semantic space also supports its effectiveness. 1

Man Is To Computer Programmer As Woman Is To Homemaker? Debiasing Word Embeddings

Abstract: The blind application of machine learning runs the risk of amplifying biases present in data. Such a danger is facing us with word embedding, a popular framework to represent text data as vectors which has been used in many machine learning and natural language processing tasks. We show that even word embeddings trained on Google News articles exhibit female/male gender stereotypes to a disturbing extent. This raises concerns because their widespread use, as we describe, often tends to amplify these biases. Geometrically, gender bias is first shown to be captured by a direction in the word embedding. Second, gender neutral words are shown to be linearly separable from gender definition words in the word embedding. Using these properties, we provide a methodology for modifying an embedding to remove gender stereotypes, such as the association between the words receptionist and female, while maintaining desired associations such as between the words queen and female. Using crowd-worker evaluation as well as standard benchmarks, we empirically demonstrate that our algorithms significantly reduce gender bias in embeddings while preserving the its useful properties such as the ability to cluster related concepts and to solve analogy tasks. The resulting embeddings can be used in applications without amplifying gender bias.

attribute accuracies
common gender prototypes


A typical paradigm in this group is Vanilla Split Learning #REFR , shown in Figure 2 , which partitions the original model into two parts, namely the server model and the client model.

In this approach, the client trains the model locally using its own data and then transmits the forward-propagated data, also known as the "Smashed Data," to the server.

The server, in turn, uses this data to update the server model and sends the backward-propagated gradients back to the client for updating the client model.

It is important to note that in this method, the clients are trained in a synchronous way, i.e., each client must wait for the previous client to complete a training round before interacting with the server.

This paradigm can be further divided into two setups: (1) centralized setup: the client uploads the model-weight-containing files to the server or a trusted third party, and the next client downloads the model weights from that server and trains the model, as shown in Figure 2 (a).

(2) the peer-to-peer (P2P) setup: the server sends the address of the previous trained client to the current client for transferring the model weights, as shown in Figure 2 (b).

Slperf: A Unified Framework For Benchmarking Split Learning

Abstract: Data privacy concerns has made centralized training of data, which is scattered across silos, infeasible, leading to the need for collaborative learning frameworks. To address that, two prominent frameworks emerged, i.e., federated learning (FL) and split learning (SL). While FL has established various benchmark frameworks and research libraries, SL currently lacks a unified library despite its diversity in terms of label sharing, model aggregation, and cut layer choice. This lack of standardization makes comparing SL paradigms difficult. To address this, we propose SLPerf, a unified research framework and open research library for SL, and conduct extensive experiments on four widely-used datasets under both IID and Non-IID data settings. Our contributions include a comprehensive survey of recently proposed SL paradigms, a detailed benchmark comparison of different SL paradigms in different situations, and rich engineering takeaway messages and research insights for improving SL paradigms. SLPerf can facilitate SL algorithm development and fair performance comparisons.

Distributed Learning Of Deep Neural Network Over Multiple Agents

Abstract: In domains such as health care and finance, shortage of labeled data and computational resources is a critical issue while developing machine learning algorithms. To address the issue of labeled data scarcity in training and deployment of neural network-based systems, we propose a new technique to train deep neural networks over several data sources. Our method allows for deep neural networks to be trained using data from multiple entities in a distributed fashion. We evaluate our algorithm on existing datasets and show that it obtains performance which is similar to a regular neural network trained on a single machine. We further extend it to incorporate semi-supervised learning when training with few labeled samples, and analyze any security concerns that may arise. Our algorithm paves the way for distributed training of deep neural networks in data sensitive applications when raw data may not be shared directly.


An Algorithm That Computes Canonical Forms Of Psd Factorizations

For this, we now formulate the task of computing canonical forms of PSD factorizations as an optimization problem, where the variables are two sets of matrices {C x } and {D y } belonging to the positive semidefinite cone: This optimization problem is non-convex and NP-hard #REFR .

According to Theorem 2.2, to determine whether or not |ψ can produce P by local operations only, one needs to characterize all possible canonical forms of PSD factorizations for P , and find out whether one of them is consistent with |ψ .

In the current section, we propose an algorithm to compute canonical forms of PSD factorizations for an arbitrary correlation P .

Since the constraints and objective function are polynomial functions of the variables, some established methods such as quantifier elimination #OTHEREFR and Lasserre's hierarchy #OTHEREFR could be employed to solve the optimization problem, albeit with exponential running time.

To obtain a heuristic approach for this problem, we fix one of the two sets of matrices to simplify it to a convex optimization problem, similar to the algorithm in #OTHEREFR for computing PSD factorization.

We accordingly develop an algorithm that computes canonical forms of PSD factorizations by alternately optimizing over {C x } and {D y }. The pseudo-code is represented in Algorithm 1. where {D y } are supposed to be fixed.

We use CVXPY to solve these subproblems #OTHEREFR , and the pseudocode is given in Algorithm 2. We now see a nontrivial application of this algorithm.

Suppose we need to determine whether a pure state |ψ can generate a target classical correlation P .

The Generations Of Classical Correlations Via Quantum Schemes

Abstract: Suppose two separated parties, Alice and Bob, share a bipartite quantum state or a classical correlation called a seed, and they try to generate a target classical correlation by performing local quantum or classical operations on the seed, i.e., any communications are not allowed. We consider the following fundamental problem about this setting: whether Alice and Bob can use a given seed to generate a target classical correlation. We show that this problem has rich mathematical structures. Firstly, we prove that even if the seed is a pure bipartite state, the above decision problem is already NP-hard and a similar conclusion can also be drawn when the seed is also a classical correlation, implying that this problem is hard to solve generally. Furthermore, we prove that when the seed is a pure quantum state, solving the problem is equivalent to finding out whether the target classical correlation has some canonical form of positive semi-definite factorizations that matches the seed pure state, revealing an interesting connection between the current problem and optimization theory. Based on this observation and other insights, we give several necessary conditions where the seed pure state has to satisfy to generate the target classical correlation, and it turns out that these conditions can also be generalized to the case that the seed is a mixed quantum state. Lastly, since canonical forms of positive semi-definite factorizations play a crucial role in solving the problem, we develop an algorithm that can compute them for an arbitrary classical correlation, which has decent performance on the cases we test.

The Complexity Of Positive Semidefinite Matrix Factorization

Abstract: Let $A$ be an $m\times n$ matrix with nonnegative real entries. The PSD rank of $A$ is the smallest integer $k$ for which there exist $k\times k$ real PSD matrices $B_1,\ldots,B_m$, $C_1,\ldots,C_n$ satisfying $A(i|j)=\operatorname{tr}(B_iC_j)$ for all $i,j$. This paper determines the computational complexity status of the PSD rank. Namely, we show that the problem of computing this function is polynomial-time equivalent to the existential theory of the reals.


Accuracy And Forgetting Rate Results And Analysis

The performance of the multi-head method HAT #REFR using task-id prediction is only 62.68, which is lower than many other baselines.

Its average accuracy over Accuracy performances of the baselines and our method MORE with smaller memory sizes.

We reduce the size of the memory buffer by half.

The new sizes are 100, 1000, 1000 for CIFAR10, CIFAR100, and Tiny-ImageNet.

Numbers in bold are the best results in each column.

all the datasets is 68.25, which is still poorer than our method's performance of 71.59.

Its performance is particularly low in experiments where the number of classes per task is small.

For instance, its accuracy on C100-20T is 56.72, much lower than our method of 70.53 trained based on OOD detection. Accuracy with Smaller Memory Sizes.

For all the datasets, we run additional experiments with half of the original memory size and show that our method is even stronger with a smaller memory.

The new memory sizes are 100, 1000, and 1000 for CIFAR-10, CIFAR-100, and Tiny-ImageNet, respectively.

Table 6 shows that MORE has experienced almost no performance drop with the reduced memory size while the memory-based baselines suffer from major performance reduction.

Open-World Continual Learning: Unifying Novelty Detection And Continual Learning

Abstract: As AI agents are increasingly used in the real open world with unknowns or novelties, they need the ability to (1) recognize objects that (i) they have learned and (ii) detect items that they have not seen or learned before, and (2) learn the new items incrementally to become more and more knowledgeable and powerful. (1) is called novelty detection or out-of-distribution (OOD) detection and (2) is called class incremental learning (CIL), which is a setting of continual learning (CL). In existing research, OOD detection and CIL are regarded as two completely different problems. This paper theoretically proves that OOD detection actually is necessary for CIL. We first show that CIL can be decomposed into two sub-problems: within-task prediction (WP) and task-id prediction (TP). We then prove that TP is correlated with OOD detection. The key theoretical result is that regardless of whether WP and OOD detection (or TP) are defined explicitly or implicitly by a CIL algorithm, good WP and good OOD detection are necessary and sufficient conditions for good CIL, which unifies novelty or OOD detection and continual learning (CIL, in particular). A good CIL algorithm based on our theory

Overcoming Catastrophic Forgetting With Hard Attention To The Task

Abstract: Catastrophic forgetting occurs when a neural network loses the information learned in a previous task after training on subsequent tasks. This problem remains a hurdle for artificial intelligence systems with sequential learning capabilities. In this paper, we propose a task-based hard attention mechanism that preserves previous tasks' information without affecting the current task's learning. A hard attention mask is learned concurrently to every task, through stochastic gradient descent, and previous masks are exploited to condition such learning. We show that the proposed mechanism is effective for reducing catastrophic forgetting, cutting current rates by 45 to 80%. We also show that it is robust to different hyperparameter choices, and that it offers a number of monitoring capabilities. The approach features the possibility to control both the stability and compactness of the learned knowledge, which we believe makes it also attractive for online learning or network compression applications.


Four members of our team write down the corresponding sentences and paragraphs with their own writing habits and imagination by observing the images in UA-DETRAC #REFR .

All volunteers are welleducated and have adequate English linguistic knowledge.

Thirdly, we insert the target vehicles with their attributes into these patterns by our sentence auto-generation framework.

1 presents, the corpus with complex context indicates narrative long sentences or paragraphs with persons' subjective emotions and imagination.

Instead of generating a corpus with simple context by combining labor and computers, a corpus with complex context is made by human beings only.

Findvehicle And Vehiclefinder: A Ner Dataset For Natural Language-Based Vehicle Retrieval And A Keyword-Based Cross-Modal Vehicle Retrieval System

Abstract: Natural language (NL) based vehicle retrieval is a task aiming to retrieve a vehicle that is most consistent with a given NL query from among all candidate vehicles. Because NL query can be easily obtained, such a task has a promising prospect in building an interactive intelligent traffic system (ITS). Current solutions mainly focus on extracting both text and image features and mapping them to the same latent space to compare the similarity. However, existing methods usually use dependency analysis or semantic role-labelling techniques to find keywords related to vehicle attributes. These techniques may require a lot of pre-processing and post-processing work, and also suffer from extracting the wrong keyword when the NL query is complex. To tackle these problems and simplify, we borrow the idea from named entity recognition (NER) and construct FindVehicle, a NER dataset in the traffic domain. It has 42.3k labelled NL descriptions of vehicle tracks, containing information such as the location, orientation, type and colour of the vehicle. FindVehicle also adopts both overlapping entities and fine-grained entities to meet further requirements. To verify its effectiveness, we propose a baseline NL-based vehicle retrieval model called VehicleFinder. Our experiment shows that by using text encoders pre-trained by FindVehicle, VehicleFinder achieves 87.7% precision and 89.4% recall when retrieving a target vehicle by text command on our homemade dataset based on UA-DETRAC [1]. From loading the command into VehicleFinder to identifying whether the target vehicle is consistent with the command, the time cost is 279.35 ms on one ARM v8.2 CPU and 93.72 ms on one RTX A4000 GPU, which is much faster than the Transformer-based system. The dataset is open-source via the link https://github.com/GuanRunwei/FindVehicle, and the implementation can be found via the link https://github.com/GuanRunwei/VehicleFinder-CTIM.

Ua-Detrac: A New Benchmark And Protocol For Multi-Object Detection And Tracking

Abstract: Effective multi-object tracking (MOT) methods have been developed in recent years for a wide range of applications including visual surveillance and behavior understanding. Existing performance evaluations of MOT methods usually separate the tracking step from the detection step by using one single predefined setting of object detection for comparisons. In this work, we propose a new University at Albany DEtection and TRACking (UA-DETRAC) dataset for comprehensive performance evaluation of MOT systems especially on detectors. The UA-DETRAC benchmark dataset consists of 100 challenging videos captured from real-world traffic scenes (over 140,000 frames with rich annotations, including illumination, vehicle type, occlusion, truncation ratio, and vehicle bounding boxes) for multi-object detection and tracking. We evaluate complete MOT systems constructed from combinations of state-of-the-art object detection and tracking methods. Our analysis shows the complex effects of detection accuracy on MOT system performance. Based on these observations, we propose effective and informative evaluation metrics for MOT systems that consider the effect of object detection for comprehensive performance analysis.


After the first prediction by a monocular depth estimation #REFR , which contains sharp edges on object borders and small details, depth maps get more blurred as compared to the previous iterations.

In this section, we address the issue of depth blurriness, which occurs when edges and high-frequency information are lost during the TTT process. As an example of this problem, in Fig.

2 , the depth map changes with each iteration after 0, 1, and 2 epochs.

During the process, we aim to preserve the high-frequency information that is present on the initial depth maps (Fig. 2,  2nd row) .

We achieve this by using gradient-based loss during training between the original depth map and the current depth estimation (at each iteration).

The proposed method can be applied to both monocular and stereo-based depth setups. In the following, we describe two gradient-based losses. Multi-scale Gradient Loss.

We propose to adopt a multiscale gradient loss #OTHEREFR as the scale-invariant gradient loss. This discrete scale-invariant gradient g is defined as:

Edge-Aware Consistent Stereo Video Depth Estimation

Abstract: Video depth estimation is crucial in various applications, such as scene reconstruction and augmented reality. In contrast to the naive method of estimating depths from images, a more sophisticated approach uses temporal information, thereby eliminating flickering and geometrical inconsistencies. We propose a consistent method for dense video depth estimation; however, unlike the existing monocular methods, ours relates to stereo videos. This technique overcomes the limitations arising from the monocular input. As a benefit of using stereo inputs, a left-right consistency loss is introduced to improve the performance. Besides, we use SLAM-based camera pose estimation in the process. To address the problem of depth blurriness during test-time training (TTT), we present an edge-preserving loss function that improves the visibility of fine details while preserving geometrical consistency. We show that our edge-aware stereo video model can accurately estimate the dense depth maps.

Towards Robust Monocular Depth Estimation: Mixing Datasets For Zero-Shot Cross-Dataset Transfer

Abstract: The success of monocular depth estimation relies on large and diverse training sets. Due to the challenges associated with acquiring dense ground-truth depth across different environments at scale, a number of datasets with distinct characteristics and biases have emerged. We develop tools that enable mixing multiple datasets during training, even if their annotations are incompatible. In particular, we propose a training objective that is invariant to changes in depth range and scale. Armed with this objective, we explore an abundant source of training data: 3D films. We demonstrate that despite pervasive inaccuracies, 3D films constitute a useful source of data that is complementary to existing training sets. We evaluate the presented approach on diverse datasets, focusing on zero-shot cross-dataset transfer: testing the generality of the learned model by evaluating it on datasets that were not seen during training. The experiments confirm that mixing data from complementary sources yields improved depth estimates, particularly on previously unseen datasets. Some results are shown in the supplementary video: this https URL


Based on LSTMs, NASA Jet Propulsion Laboratory designs an unsupervised nonparametric anomaly thresholding ap-proach for spacecraft anomaly detection #REFR ; Park et al.

Recently, deep neural network architectures have achieved leading performance on various time series anomaly detection tasks.

Especially, LSTMs and RNNs can effectively process time-series data, capturing valuable historical information for future prediction.

#OTHEREFR propose variational auto-encoders that fuse signals and reconstruct expected distribution.

Most telemetry data are multi-dimensional due to the interrelation of components in the satellite's internal structure. As illustrated in Fig.

1 , each curve corresponds to a variable (or channel) in spacecraft multivariate time series, and an anomaly in one channel of the spacecraft also causes abnormality in other channels.

For online anomaly de- #OTHEREFR utilize stochastic variable connection and planar normalizing flow for multivariate time series anomaly detection, which performs well in datasets of NASA. Zhao et al.

#OTHEREFR utilizes the graph attention method to capture temporal and variable dependency for anomaly detection. Li et al.

Spacecraft Anomaly Detection With Attention Temporal Convolution Networks

Abstract: Spacecraft faces various situations when carrying out exploration missions in complex space, thus monitoring the anomaly status of spacecraft is crucial to the development of the aerospace industry. The time series telemetry data generated by on-orbit spacecraft contains important information about the status of spacecraft. However, traditional domain knowledge-based spacecraft anomaly detection methods are not effective due to high dimensionality and complex correlation among variables. In this work, we propose an anomaly detection framework for spacecraft multivariate time-series data based on temporal convolution networks (TCNs). First, we employ dynamic graph attention to model the complex correlation among variables and time series. Second, temporal convolution networks with parallel processing ability are used to extract multidimensional features for the downstream prediction task. Finally, many potential anomalies are detected by the best threshold. Experiments on real NASA SMAP/MSL spacecraft datasets show the superiority of our proposed model with respect to state-of-the-art methods.

Detecting Spacecraft Anomalies Using Lstms And Nonparametric Dynamic Thresholding

Abstract: As spacecraft send back increasing amounts of telemetry data, improved anomaly detection systems are needed to lessen the monitoring burden placed on operations engineers and reduce operational risk. Current spacecraft monitoring systems only target a subset of anomaly types and often require costly expert knowledge to develop and maintain due to challenges involving scale and complexity. We demonstrate the effectiveness of Long Short-Term Memory (LSTMs) networks, a type of Recurrent Neural Network (RNN), in overcoming these issues using expert-labeled telemetry anomaly data from the Soil Moisture Active Passive (SMAP) satellite and the Mars Science Laboratory (MSL) rover, Curiosity. We also propose a complementary unsupervised and nonparametric anomaly thresholding approach developed during a pilot implementation of an anomaly detection system for SMAP, and offer false positive mitigation strategies along with other key improvements and lessons learned during development.


I. Introduction Most Modern Synthetic Aperture Radar (Sar) Automatic

SSL has shown great promise for transfer and few-shot learning in the natural imagery domain #REFR , but has not been studied widely in the context of SAR ATR.

In Stage 2, for an arbitrary downstream (M-way, N-shot) few-shot learning (FSL) task we train a lightweight classifier using the global model as a fixed feature extractor.

A key part of our methodology is how we train the global model.

Importantly, we do not assume to have labeled data in this step, which eliminates the option for a meta-learning approach #OTHEREFR , #OTHEREFR .

To learn quality representations we instead aggregate publicly available data from recent papers and leverage a modern unsupervised learning paradigm called Self-Supervised Learning (SSL).

Specifically, we use the SimCLR #OTHEREFR method to represent the SSL class of algorithms in this work.

Our hypothesis is that these modern algorithms can learn a highly flexible / expressive / useful / transferable feature extractor from unlabeled SAR data sourced from different sensors, imaging modes, polarizations, resolutions, and target types.

Further, the unlabeled data does not necessarily have to represent the anticipated downstream conditions.

This is contrary to several lines of concurrent research whose goal is to generate highfidelity synthetic imagery as close to the downstream task conditions as possible for supervised pre-training #OTHEREFR .

The second key part of our methodology is how we improve robustness to OOD inputs.

This is accomplished in the classifier training stage and given our stated operating environment it is nearly a "free-lunch." Practically, we can use the SSL model's pretraining dataset as an OOD dataset for outlier arXiv:2303.10800v1 [cs.CV] 20 Mar 2023 exposure (OE) training #OTHEREFR , #OTHEREFR .

A Global Model Approach To Robust Few-Shot Sar Automatic Target Recognition

Abstract: In real-world scenarios, it may not always be possible to collect hundreds of labeled samples per class for training deep learning-based SAR Automatic Target Recognition (ATR) models. This work specifically tackles the few-shot SAR ATR problem, where only a handful of labeled samples may be available to support the task of interest. Our approach is composed of two stages. In the first, a global representation model is trained via self-supervised learning on a large pool of diverse and unlabeled SAR data. In the second stage, the global model is used as a fixed feature extractor and a classifier is trained to partition the feature space given the few-shot support samples, while simultaneously being calibrated to detect anomalous inputs. Unlike competing approaches which require a pristine labeled dataset for pretraining via meta-learning, our approach learns highly transferable features from unlabeled data that have littleto-no relation to the downstream task. We evaluate our method in standard and extended MSTAR operating conditions and find it to achieve high accuracy and robust out-of-distribution detection in many different few-shot settings. Our results are particularly significant because they show the merit of a global model approach to SAR ATR, which makes minimal assumptions, and provides many axes for extendability.

Rethinking Few-Shot Image Classification: A Good Embedding Is All You Need?

Abstract: The focus of recent meta-learning research has been on the development of learning algorithms that can quickly adapt to test time tasks with limited data and low computational cost. Few-shot learning is widely used as one of the standard benchmarks in meta-learning. In this work, we show that a simple baseline: learning a supervised or self-supervised representation on the meta-training set, followed by training a linear classifier on top of this representation, outperforms state-of-the-art few-shot learning methods. An additional boost can be achieved through the use of self-distillation. This demonstrates that using a good learned embedding model can be more effective than sophisticated meta-learning algorithms. We believe that our findings motivate a rethinking of few-shot image classification benchmarks and the associated role of meta-learning algorithms. Code is available at: http://github.com/WangYueFt/rfs/.


Next, we consider the transmission of the encoded signal over the wireless channels after the encoding and normalization operations, i.e., y = hx + n, where the communication channels h are integrated as non-trainable layers into the whole deep JSCC framework to enable end-to-end training #REFR .

x ∈ R 2b of e(•, θ) are denoted asx = e(s, θ).

Then we combine (reshape)x into b complexvalued symbols to form the encoded signal x. After encoding the task-relevant semantic signal

x from the transmitted signal s, we normalize it aŝ

such that the transmit signalx satisfies the average transmit power constraint in (1).

Finally, we discuss the processing of the received signal y at the receiver to perform image recovery and classification tasks simultaneously, which is different from semantic communication frameworks dedicatedly designed for only one task in previous works #OTHEREFR - #OTHEREFR , #OTHEREFR - #OTHEREFR .

First, the receiver applies channel equalization according to #OTHEREFR Extract semantic features via e(S, θ).

Semantic Communications For Image Recovery And Classification Via Deep Joint Source And Channel Coding

Abstract: With the recent advancements in edge artificial intelligence (AI), future sixth-generation (6G) networks need to support new AI tasks such as classification and clustering apart from data recovery. Motivated by the success of deep learning, the semantic-aware and task-oriented communications with deep joint source and channel coding (JSCC) have emerged as new paradigm shifts in 6G from the conventional data-oriented communications with separate source and channel coding (SSCC). However, most existing works focused on the deep JSCC designs for one task of data recovery or AI task execution independently, which cannot be transferred to other unintended tasks. Differently, this paper investigates the JSCC semantic communications to support multi-task services, by performing the image data recovery and classification task execution simultaneously. First, we propose a new end-to-end deep JSCC framework by unifying the coding rate reduction maximization and the mean square error (MSE) minimization in the loss function. Here, the coding rate reduction maximization facilitates the learning of Part of this paper has been submitted to IEEE Global Communications Conference (GLOBECOM) 2023 [1].

Deep Learning Based End-To-End Wireless Communication Systems With Conditional Gan As Unknown Channel

Abstract: In this article, we develop an end-to-end wireless communication system using deep neural networks (DNNs), in which DNNs are employed to perform several key functions, including encoding, decoding, modulation, and demodulation. However, an accurate estimation of instantaneous channel transfer function, i.e., channel state information (CSI), is needed in order for the transmitter DNN to learn to optimize the receiver gain in decoding. This is very much a challenge since CSI varies with time and location in wireless communications and is hard to obtain when designing transceivers. We propose to use a conditional generative adversarial net (GAN) to represent channel effects and to bridge the transmitter DNN and the receiver DNN so that the gradient of the transmitter DNN can be back-propagated from the receiver DNN. In particular, a conditional GAN is employed to model the channel effects in a data-driven way, where the received signal corresponding to the pilot symbols is added as a part of the conditioning information of the GAN. To address the curse of dimensionality when the transmit symbol sequence is long, convolutional layers are utilized. From the simulation results, the proposed method is effective on additive white Gaussian noise (AWGN) channels, Rayleigh fading channels, and frequency-selective channels, which opens a new door for building data-driven DNNs for end-to-end communication systems.


This convolutional network architecture introduced by #REFR processes audio sample as one-dimensional raw waveforms in time domain.

• Noise with signal-to-noise-ratio between 10 −4 and 10 −1 .

• Bandpass filter with cutoff frequencies in the intervals , = [1400 ; 4000 ] and , = [500 ; 1200 ] for a low pass (LP) and high pass (HP) filter, respectively.

It includes four convolutional layers and contains 2.5×10 6 parameters in total.

We train 1DCNN on the UrbanSound8k audio event classification task and achieve a training set accuracy of 75.22 ± 0.44%, and test set accuracy of 54.94 ± 5.19% over a 10-fold cross-validation.

This is a convolutional neural network, that processes audio samples as two-dimensional logmel-spectrograms in the time-frequency domain.

It is built on the architecture of Mobilenet V1 #OTHEREFR .

Xai-Based Comparison Of Input Representations For Audio Event Classification

Abstract: Deep neural networks are a promising tool for Audio Event Classification. In contrast to other data like natural images, there are many sensible and non-obvious representations for audio data, which could serve as input to these models. Due to their black-box nature, the effect of different input representations has so far mostly been investigated by measuring classification performance. In this work, we leverage eXplainable AI (XAI), to understand the underlying classification strategies of models trained on different input representations. Specifically, we compare two model architectures with regard to relevant input features used for Audio Event Detection: one directly processes the signal as the raw waveform, and the other takes in its time-frequency spectrogram representation. We show how relevance heatmaps obtained via "Siren"Layer-wise Relevance Propagation uncover representation-dependent decision strategies. With these insights, we can make a well-informed decision about the best input representation in terms of robustness and representativity and confirm that the model's classification strategies align with human requirements.

End-To-End Environmental Sound Classification Using A 1D Convolutional Neural Network

Abstract: In this paper, we present an end-to-end approach for environmental sound classification based on a 1D Convolution Neural Network (CNN) that learns a representation directly from the audio signal. Several convolutional layers are used to capture the signal's fine time structure and learn diverse filters that are relevant to the classification task. The proposed approach can deal with audio signals of any length as it splits the signal into overlapped frames using a sliding window. Different architectures considering several input sizes are evaluated, including the initialization of the first convolutional layer with a Gammatone filterbank that models the human auditory filter response in the cochlea. The performance of the proposed end-to-end approach in classifying environmental sounds was assessed on the UrbanSound8k dataset and the experimental results have shown that it achieves 89% of mean accuracy. Therefore, the propose approach outperforms most of the state-of-the-art approaches that use handcrafted features or 2D representations as input. Furthermore, the proposed approach has a small number of parameters compared to other architectures found in the literature, which reduces the amount of data required for training.


We use the Open3D #REFR library and the location of the points to create a point cloud.

We follow the four steps below to do the same. Fig.

3 illustrates how a mesh is generated from RGB-D information from HoloLens.

We create a set of point clouds from each depth image in the dataset.

For this, we first undistort the images and for each pixel in the undistorted image, we use the intrinsic parameters such as the focal length and the principal point and the depth value to calculate the 3D position of the point corresponding to that pixel in the camera's coordinate system.

Since the point cloud was noisy, we tried to remove them using a radius outlier filter and a statistical outlier filter from Open3D.

Using these two filters, we were effectively able to reduce the noise.

Depth cameras generally do not work well with highly reflective, transparent, or black surfaces and hence some of the points were very far away from or very close to the camera.

This is an inherent problem of the depth sensor as it uses time-of-flight to measure the distance.

To overcome this, we used the fact that the data was recorded inside a room and away from various artefacts.

Image Stabilization For Hololens Camera In Remote Collaboration

Abstract: With the advent of new technologies, Augmented Reality (AR) has become an effective tool in remote collaboration. Narrow field-of-view (FoV) and motion blur can offer an unpleasant experience with limited cognition for remote viewers of AR headsets. In this article, we propose a two-stage pipeline to tackle this issue and ensure a stable viewing experience with a larger FoV. The solution involves an offline 3D reconstruction of the indoor environment, followed by enhanced rendering using only the live poses of AR device. We experiment with and evaluate the two different 3D reconstruction methods, RGB-D geometric approach and Neural Radiance Fields (NeRF), based on their data requirements, reconstruction quality, rendering, and training times. The generated sequences from these methods had smoother transitions and provided a better perspective of the environment. The geometry-based enhanced FoV method had better renderings as it lacked blurry outputs making it better than the other attempted approaches. Structural Similarity Index (SSIM) and Peak Signal to Noise Ratio (PSNR) metrics were used to quantitatively show that the rendering quality using the geometry-based en

Open3D: A Modern Library For 3D Data Processing

Abstract: Open3D is an open-source library that supports rapid development of software that deals with 3D data. The Open3D frontend exposes a set of carefully selected data structures and algorithms in both C++ and Python. The backend is highly optimized and is set up for parallelization. Open3D was developed from a clean slate with a small and carefully considered set of dependencies. It can be set up on different platforms and compiled from source with minimal effort. The code is clean, consistently styled, and maintained via a clear code review mechanism. Open3D has been used in a number of published research projects and is actively deployed in the cloud. We welcome contributions from the open-source community.


Alongside looking at pre-processing techniques #REFR , we find that an interesting direction for defensive research could be to increase the resilience of code models by designing more robust feature mapping processes.

Our experiments suggest that, depending on the characteristics of a given model and pair of functions, the success of an attack may be affected by factors like the initial difference in code size and CFG topology, among others.

In this respect, it could be interesting to explore how to alternate our dead-branch addition perturbation, for example, with the insertion of dead fragments within existing blocks.

We believe both limitations could be addressed in future work with implementation effort, whereas the main goal of this paper was to show that adversarial attacks against binary similarity systems are a concrete possibility.

For defensive attempts, we could rely on off-the-shelf obfuscation methods (Section 3.3).

To enhance our attacks, we could explore more complex patching implementation strategies based on binary rewriting or a modified compiler back-end.

Adversarial Attacks Against Binary Similarity Systems

Abstract: In recent years, binary analysis gained traction as a fundamental approach to inspect software and guarantee its security. Due to the exponential increase of devices running software, much research is now moving towards new autonomous solutions based on deep learning models, as they have been showing state-of-the-art performances in solving binary analysis problems. One of the hot topics in this context is binary similarity, which consists in determining if two functions in assembly code are compiled from the same source code. However, it is unclear how deep learning models for binary similarity behave in an adversarial context. In this paper, we study the resilience of binary similarity models against adversarial examples, showing that they are susceptible to both targeted and untargeted attacks (w.r.t. similarity goals) performed by black-box and white-box attackers. In more detail, we extensively test three current state-of-the-art solutions for binary similarity against two black-box greedy attacks, including a new technique that we call Spatial Greedy, and one white-box attack in which we repurpose a gradient-guided strategy used in attacks to image classifiers.

Intriguing Properties Of Adversarial Ml Attacks In The Problem Space

Abstract: Recent research efforts on adversarial ML have investigated problem-space attacks, focusing on the generation of real evasive objects in domains where, unlike images, there is no clear inverse mapping to the feature space (e.g., software). However, the design, comparison, and real-world implications of problem-space attacks remain underexplored. This paper makes two major contributions. First, we propose a general formalization for adversarial ML evasion attacks in the problem-space, which includes the definition of a comprehensive set of constraints on available transformations, preserved semantics, absent artifacts, and plausibility. We shed light on the relationship between feature space and problem space, and we introduce the concept of side-effect features as the by-product of the inverse feature-mapping problem. This enables us to define and prove necessary and sufficient conditions for the existence of problem-space attacks. We further demonstrate the expressive power of our formalization by using it to describe several attacks from related literature across different domains. Second, building on our general formalization, we propose a novel problem-space attack on Android malware that overcomes past limitations in terms of semantics and artifacts. Experiments on a dataset with 170K Android apps from 2017 and 2018 show the practical feasibility of evading a state-of-the-art malware classifier, DREBIN, along with its hardened version, Sec-SVM. Our results demonstrate that "adversarial-malware as a service" is a realistic threat, as we automatically generate thousands of realistic and inconspicuous adversarial applications at scale, where on average it takes only a few minutes to generate an adversarial app. Our formalization of problem-space attacks paves the way to more principled research in this domain.


We notice that our implementation of FBCSP obtains a lower score than what was reported by the authors of #REFR on the same dataset.

We decided to use the High-Gamma Dataset in our study even though it was not specifically created to learn embeddings or benchmark embedding approaches.

While its high number of trials per subject (around 1000) allows for the benchmarking of various scenarios, results need to be judged in the light of the mediocre number of 14 subjects in this dataset.

Clear advantages of this dataset are that its authors #OTHEREFR

[2017] were successful in training deep models on it, and that it has been used in follow-up publications.

Performance of the FBCSP baseline method.

We qualify this point by reminding that we use different dataset preprocessings.

We low-pass filter to frequencies up to 40 Hz before downsampling to 128 Hz, while they allowed up to 128 Hz frequencies before downsampling to 256 Hz.

In addition, Schirrmeister and colleagues used a larger trial time window (from 0.5 s to 4 s relative to trial start, while we have removed the first 0.5 s of every trial.

In addition, we can not exclude that our implementation of FBCSP may differ from theirs on some details.

Balancing the influence of subject and task classes The results of the embedding models on the cross-subject training (cf. Result 1) revealed a slightly unexpected finding.

An Embedding For Eeg Signals Learned Using A Triplet Loss

Abstract: Neurophysiological time series recordings like the electroencephalogram (EEG) or local field potentials are obtained from multiple sensors. They can be decoded by machine learning models in order to estimate the ongoing brain state of a patient or healthy user. In a brain-computer interface (BCI), this decoded brain state information can be used with minimal time delay to either control an application, e.g., for communication or for rehabilitation after stroke, or to passively monitor the ongoing brain state of the subject, e.g., in a demanding work environment. A specific challenge in such decoding tasks is posed by the small dataset sizes in BCI compared to other domains of machine learning like computer vision or natural language processing. A possibility to tackle classification or regression problems in BCI despite small training data sets is through transfer learning, which utilizes data from other sessions, subjects or even datasets to train a model. In this exploratory study, we propose novel domain-specific embeddings for neurophysiological data. Our approach is based on metric learning and builds upon the recently proposed ladder loss. Using embeddings allowed us to benefit, both from the good generalisation abilities and robustness of deep learning and from the fast training of classical machine learning models for subject-specific calibration. In offline analyses using EEG data of 14 subjects, we tested the embeddings' feasibility and compared their efficiency with state-of-the-art deep learning models and conventional machine learning pipelines. For within-subject classification of motor tasks, our data suggests that embeddings significantly improve the decoding accuracy compared to an established baseline model. In between-subjects transfer scenarios, pre-trained embeddings allow for an extremely fast fine-tuning of a classifier for a novel subject, even if only a few trials per class are available, which currently can not be achieved with regular deep learning models. And in the scenarios where one subject is completely set aside for testing without specific calibration, the embeddings were not competitive. In summary, we propose the use of metric learning to obtain pre-trained embeddings of EEG-BCI data as a means to incorporate domain knowledge and to reach competitive performance on novel subjects with minimal calibration requirements.

Deep Learning With Convolutional Neural Networks For Decoding And Visualization Of Eeg Pathology

Abstract: We apply convolutional neural networks (ConvNets) to the task of distinguishing pathological from normal EEG recordings in the Temple University Hospital EEG Abnormal Corpus. We use two basic, shallow and deep ConvNet architectures recently shown to decode task-related information from EEG at least as well as established algorithms designed for this purpose. In decoding EEG pathology, both ConvNets reached substantially better accuracies (about 6% better, ~85% vs. ~79%) than the only published result for this dataset, and were still better when using only 1 minute of each recording for training and only six seconds of each recording for testing. We used automated methods to optimize architectural hyperparameters and found intriguingly different ConvNet architectures, e.g., with max pooling as the only nonlinearity. Visualizations of the ConvNet decoding behavior showed that they used spectral power changes in the delta (0-4 Hz) and theta (4-8 Hz) frequency range, possibly alongside other features, consistent with expectations derived from spectral analysis of the EEG data and from the textual medical reports. Analysis of the textual medical reports also highlighted the potential for accuracy increases by integrating contextual information, such as the age of subjects. In summary, the ConvNets and visualization techniques used in this study constitute a next step towards clinically useful automated EEG diagnosis and establish a new baseline for future work on this topic.


The case studies were conducted with Adam optimizer in a Pytorch environment #REFR using an Ampere A-100 GPU.

We tuned the hyperparameters, including batch size, sequence length and learning rate, of SA-MGCRN model for each instance (i.e., each date and each data delivery data scenario).

We adopted an early-stopping technique to prevent the model from overfitting and improve the training efficiency.

If the trained model does not have an accuracy improvement during the last 200 training epochs, we consider the model has already converged.

In our case study, we update (i.e., retune) SA-MGCRN 14 times (from October 23, 2019 to November 6, 2019) in total.

The testing set is set as the travel demand on the following day after each model updates.

Situational-Aware Multi-Graph Convolutional Recurrent Network (Sa-Mgcrn) For Travel Demand Forecasting During Wildfires

Abstract: Real-time forecasting of travel demand during wildfire evacuations is crucial for emergency managers and transportation planners to make timely and better-informed decisions. However, few studies focus on accurate travel demand forecasting in large-scale emergency evacuations. Therefore, this study develops and tests a new methodological framework for modeling trip generation in wildfire evacuations by using (a) large-scale GPS data generated by mobile devices and (b) state-of-the-art AI technologies. The proposed methodology aims at forecasting evacuation trips and other types of trips. Based on the travel demand inferred from the GPS data, we develop a new deep learning model, i.e., Situational-Aware Multi-Graph Convolutional Recurrent Network (SA-MGCRN), along with a model updating scheme to achieve real-time forecasting of travel demand during wildfire evacuations. The proposed methodological framework is tested in this study for a real-world case study: the 2019 Kincade Fire in Sonoma County, CA. The results show that SA-MGCRN significantly outperforms all the selected state-of-the-art benchmarks in terms of prediction performance. Our finding suggests that the most important model components of SA-MGCRN are evacuation order/warning information, proximity to fire, and population change, which are consistent with behavioral theories and empirical findings.

Pytorch: An Imperative Style, High-Performance Deep Learning Library

Abstract: Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it provides an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. ::: In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. ::: We demonstrate the efficiency of individual subsystems, as well as the overall speed of PyTorch on several common benchmarks.