Since various users have different sizes of their preference sets, we follow the strategy proposed in #REFR , i.e., using a fixed-size sampler and the size of the sampler is an adjustable parameter.

In the model RippleNet #OTHEREFR , the user's preferences are simulated by the multistage propagation of user history, and we used this setting, Figure 4 shows a simple example of a 3-stage preference set of a user whose click history only has the Forrest Gump movie.

We assume that a user has only watched one movie, Forrest Gump, then the user's 1-stage preference set P 1 includes four entities: Drama, U.S, Tom Hanks, and Robert Zemeckis.

Similarly, the entity contained in the hop2 part of the figure is the user's 2-stage preference set P 2 .

In reality, the number of items in the interaction history of a user is variable.

Ripple Knowledge Graph Convolutional Networks For Recommendation Systems

Abstract: Using knowledge graphs to assist deep learning models in making recommendation decisions has recently been proven to effectively improve the model's interpretability and accuracy. This paper introduces an end-toend deep learning model, named RKGCN, which dynamically analyses each user's preferences and makes a recommendation of suitable items. It combines knowledge graphs on both the item side and user side to enrich their representations to maximize the utilization of the abundant information in knowledge graphs. RKGCN is able to offer more personalized and relevant recommendations in three different scenarios. The experimental results show the superior effectiveness of our model over 5 baseline models on three real-world datasets including movies, books, and music.

Knowledge Graph Convolutional Networks For Recommender Systems

Abstract: To alleviate sparsity and cold start problem of collaborative filtering based recommender systems, researchers and engineers usually collect attributes of users and items, and design delicate algorithms to exploit these additional information. In general, the attributes are not isolated but connected with each other, which forms a knowledge graph (KG). In this paper, we propose Knowledge Graph Convolutional Networks (KGCN), an end-to-end framework that captures inter-item relatedness effectively by mining their associated attributes on the KG. To automatically discover both high-order structure information and semantic information of the KG, we sample from the neighbors for each entity in the KG as their receptive field, then combine neighborhood information with bias when calculating the representation of a given entity. The receptive field can be extended to multiple hops away to model high-order proximity information and capture users' potential long-distance interests. Moreover, we implement the proposed KGCN in a minibatch fashion, which enables our model to operate on large datasets and KGs. We apply the proposed model to three datasets about movie, book, and music recommendation, and experiment results demonstrate that our approach outperforms strong recommender baselines.


Chinese pre-trained large models based on the Transformer #REFR architecture are trained using massive textual corpora, including Chinese encyclopedic data, a large number of Chinese e-books, and numerous Chinese websites.

With the impressive debut of large models such as ChatGPT (Brockman et al., 2023) , a series of Chinese-capable large models, including ChatGLM #OTHEREFR , MOSS (Sun et al., 2023) , Wenxin Yiyan, Tongyi Qianwen, and Shangliang, have been released successively.

While there are well-established evaluation methods for large English language models, such as MMLU #OTHEREFR , there is still a lack of evaluations for large Chinese language models.

Therefore, it is imperative to introduce a scientific evaluation method for large Chinese models and provide high-quality Chinese evaluation datasets.

However, the capabilities of these models in understanding and solving problems across various domains have not been scientifically and comprehensively evaluated.

Since most large language models released recently have undergone prompt-tuning training, we provide both zero-shot and few-shot testing modes, with the latter offering five examples to the models.

The test questions are single-choice and multiple-choice, with each question potentially having one or multiple correct answers, making them more akin to human exams and increasing their difficulty.

This test covers four major categories: medicine, law, psychology, and education.

Medical questions are drawn from university medical examinations, legal questions from the National Uniform Legal Professional Qualification Examination, psychology questions from psychological counselor exams and graduate entrance exams in psychology, and education questions from the Chinese National College Entrance Examination.

Measuring Massive Multitask Chinese Understanding

Abstract: The development of large-scale Chinese language models is flourishing, yet there is a lack of corresponding capability assessments. Therefore, we propose a test to measure the multitask accuracy of large Chinese language models. This test encompasses four major domains, including medicine, law, psychology, and education, with 15 subtasks in medicine and 8 subtasks in education. We found that the best-performing models in the zero-shot setting outperformed the worstperforming models by nearly 22 percentage points on average. Across the four major domains, the average zero-shot accuracy of all models did not exceed 0.5. In the subdomains, only the GPT-3.5turbo model achieved a zero-shot accuracy of 0.703 in clinical medicine, which was the highest accuracy among all models across all subtasks. All models performed poorly in the legal domain, with the highest zero-shot accuracy reaching only 0.259. By comprehensively evaluating the breadth and depth of knowledge across multiple disciplines, this test can more accurately identify the shortcomings of the models.

Attention Is All You Need

Abstract: The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.


The performance overhead of using Apache Beam as an abstraction layer has also been observed in related research #REFR . higher resource demand compared to the other evaluated frameworks.

3b ), it appears that the resource demand of Flink increases at a steeper rate as well.

For lower load intensities (less than 5 nested groups), Flink requires fewer instances than Samza.

Worth mentioning is also the significant difference between native Apache Flink and the Flink runner of Apache Beam.

In almost all experiments, the resource demand of Apache Beam with Flink is at least twice as high.

For the more compute- intensive benchmarks UC3 and UC4, it is tremendously higher.

In this section, we take a closer look at the scalability of the Apache Beam SUTs and evaluate how scalability is affected by different configuration options.

Again, we use the private cloud environment (see Table 1 ).

In the following, we first look at the Apache Flink runner and, afterward, at the Apache Samza runner.

Benchmarking Scalability Of Stream Processing Frameworks Deployed As Event-Driven Microservices In The Cloud

Abstract: Event-driven microservices are an emerging architectural style for data-intensive software systems. In such systems, stream processing frameworks such as Apache Flink, Apache Kafka Streams, Apache Samza, Hazelcast Jet, or the Apache Beam SDK are used inside microservices to continuously process massive amounts of data in a distributed fashion. While all of these frameworks promote scalability as a core feature, there is only little empirical research evaluating and comparing their scalability. In this study, we benchmark five modern stream processing frameworks regarding their scalability using a systematic method. We conduct over 460 hours of experiments on Kubernetes clusters in the Google cloud and in a private cloud, where we deploy up to 110 simultaneously running microservice instances, which process up to one million messages per second. We find that all benchmarked frameworks exhibit approximately linear scalability as long as sufficient cloud resources are provisioned. However, the frameworks show considerable differences in the rate at which resources have to be added to cope with increasing load. Moreover, we observe that there is no clear superior framework, but the ranking of the frameworks depends on the use case. Using Apache Beam as an abstraction layer still comes at the cost of significantly higher resource requirements regardless of the use case.

Quantitative Impact Evaluation Of An Abstraction Layer For Data Stream Processing Systems

Abstract: With the demand to process ever-growing data volumes, a variety of new data stream processing frameworks have been developed. Moving an implementation from one such system to another, e.g., for performance reasons, requires adapting existing applications to new interfaces. Apache Beam addresses these high substitution costs by providing an abstraction layer that enables executing programs on any of the supported streaming frameworks. In this paper, we present a novel benchmark architecture for comparing the performance impact of using Apache Beam on three streaming frameworks: Apache Spark Streaming, Apache Flink, and Apache Apex. We find significant performance penalties when using Apache Beam for application development in the surveyed systems. Overall, usage of Apache Beam for the examined streaming applications caused a high variance of query execution times with a slowdown of up to a factor of 58 compared to queries developed without the abstraction layer. All developed benchmark artifacts are publicly available to ensure reproducible results.


For the text-conditional inpainting task, Inst-Inpaint tokenizes the text using the BERT tokenizer #REFR , and by taking these tokens as input, it learns a transformer model from scratch to obtain the instruction embeddings.

(2) Here, i refers to the text instruction and T , the transformer.

The encoder, E, is fixed during training whereas θ and T are optimized together.

Inst-Inpaint concatenates the latent code of the target image z t at timestep t, obtained from the forward diffusion process, and the source image feature E(s) to predict the latent code of the previous timestep z t−1 .

These embeddings are used in the crossattention mechanisms of the specific U-Net layers.

We trained two Inst-Inpaint models on the CLEVR and the GQA-Inpaint datasets.

The first stage model of the CLEVR model is a variational autoencoder trained from scratch on the combination of both the source and the target images of all instructions in the dataset.

Differently from the CLEVR model, GQA-Inpaint model uses a pretrained VQGAN model 1 , which is trained on Open Images dataset #OTHEREFR and has 16384 number of codebook entries.

In the second stage model of the CLEVR model, we define an eight-layer transformer to obtain the instruction embeddings and four-head cross attention to learn the relation between the instruction embedding and the activation layers for the inpainting process.

Inst-Inpaint: Instructing To Remove Objects With Diffusion Models

Abstract: Figure 1. Instructional Image Inpainting. We propose a new image inpainting method, Inst-Inpaint, which takes an image and a textual instruction as input and is able to automatically remove objects mentioned in the text, without any need for a user-supplied binary mask to locate the region of interest, as in traditional inpainting approaches. We show sample results of our method trained and tested on a new real image inpainting benchmark dataset, GQA-Inpaint, which we created for the proposed instructional image inpainting task.

Bert: Pre-Training Of Deep Bidirectional Transformers For Language Understanding

Abstract: We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications. ::: BERT is conceptually simple and empirically powerful. It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5% (7.7% point absolute improvement), MultiNLI accuracy to 86.7% (4.6% absolute improvement), SQuAD v1.1 question answering Test F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1 (5.1 point absolute improvement).

text-conditional inpainting task
BERT tokenizer


Unlike previous work in the field of natural language processing #REFR , we extend the words in a one-dimensional sequence to the pixels in a two-dimensional plane, considering their relative positions as a combined effect along both horizontal and vertical directions.

Finally, the context matrix c contextualizes the input features q so that the attention can be adjusted by the given context. Position Prior Embedding.

In remote sensing images, ground objects are spatially distributed following specific intrinsic patterns.

In particular, certain combinations or concurrences usually occur to the objects in close proximity, and the pixels in an object's vicinity may demonstrate high correlation.

These observations suggest that a pixel's awareness of the scene relies on its sensitivity to relative positions, which are embedded as,

Specifically, the encoding of relative position r ij is defined as follows,

where P ∈ R (2ξ+1)×(2ξ+1)×C is a bucket storing a set of indexed trainable vectors,

I x (i, j) = g(x i − x j ) and I y (i, j) = g(y i − y j )

represent subscripts for horizontal and vertical directions, forming two-dimensional indices for P , and g denotes an index function as follows,

Sacanet: Scene-Aware Class Attention Network For Semantic Segmentation Of Remote Sensing Images

Abstract: Spatial attention mechanism has been widely used in semantic segmentation of remote sensing images given its capability to model long-range dependencies. Many methods adopting spatial attention mechanism aggregate contextual information using direct relationships between pixels within an image, while ignoring the scene awareness of pixels (i.e., being aware of the global context of the scene where the pixels are located and perceiving their relative positions). Given the observation that scene awareness benefits context modeling with spatial correlations of ground objects, we design a scene-aware attention module based on a refined spatial attention mechanism embedding scene awareness. Besides, we present a local-global class attention mechanism to address the problem that general attention mechanism introduces excessive background noises while hardly considering the large intra-class variance in remote sensing images. In this paper, we integrate both scene-aware and class attentions to propose a scene-aware class attention network (SACANet) for semantic segmentation of remote sensing images. Experimental results on three datasets show that SACANet outperforms other state-ofthe-art methods and validate its effectiveness.

Self-Attention With Relative Position Representations

Abstract: Relying entirely on an attention mechanism, the Transformer introduced by Vaswani et al. (2017) achieves state-of-the-art results for machine translation. In contrast to recurrent and convolutional neural networks, it does not explicitly model relative or absolute position information in its structure. Instead, it requires adding representations of absolute positions to its inputs. In this work we present an alternative approach, extending the self-attention mechanism to efficiently consider representations of the relative positions, or distances between sequence elements. On the WMT 2014 English-to-German and English-to-French translation tasks, this approach yields improvements of 1.3 BLEU and 0.3 BLEU over absolute position representations, respectively. Notably, we observe that combining relative and absolute position representations yields no further improvement in translation quality. We describe an efficient implementation of our method and cast it as an instance of relation-aware self-attention mechanisms that can generalize to arbitrary graph-labeled inputs.


A family of neural processors often resort to the DeepSet architecture #REFR ) that first projects each set element independently and then pools representations together with a permutation-invariant function (e.g., sum, mean) followed by an MLP.

We elaborate on the differences of those query patterns in the following Section 6.2 and emphasize our focus on more complex intersection queries going beyond simpler path-like queries.

Query processors, as described in Section 5.2, employ different parameterizations of conjunctions as permutation-invariant set functions.

Alternatively, self-attention, can serve as a replacement of the DeepSet where set elements are weighted with the attention operation.

The other family of neural processors combine projection and intersection by processing the whole query graph with GNNs #OTHEREFR or with the Transformer over linearized query sequences (Kotnis et al., 2021; .

Geometric processors implement conjunction as the attention-based average of centroids and offsets of respective geometric objects (boxes, hyperboloids, or cones).

Probabilistic processors Figure 18 : Query operators (relation projection and intersection), corresponding SPARQL basic graph patterns (BGP), and their computation graphs. Relation Projection (left) corresponds to a triple pattern.

et al., 2022b; Yang et al., 2022a) implement intersection as a weighted sum of parametric distributions that represent queries and variables.

Neural Graph Reasoning: Complex Logical Query Answering Meets Graph Databases

Abstract: Complex logical query answering (CLQA) is a recently emerged task of graph machine learning that goes beyond simple one-hop link prediction and solves a far more complex task of multi-hop logical reasoning over massive, potentially incomplete graphs in a latent space. The task received a significant traction in the community; numerous works expanded the field along theoretical and practical axes to tackle different types of complex queries and graph modalities with efficient systems. In this paper, we provide a holistic survey of CLQA with a detailed taxonomy studying the field from multiple angles, including graph types (modality, reasoning domain, background semantics), modeling aspects (encoder, processor, decoder), supported queries (operators, patterns, projected variables), datasets, evaluation metrics, and applications. Refining the CLQA task, we introduce the concept of Neural Graph Databases (NGDBs). Extending the idea of graph databases (graph DBs), NGDB consists of a Neural Graph Storage and a Neural Graph Engine. Inside Neural Graph Storage, we design a graph store, a feature store, and further embed information in a latent embedding store using an encoder. Given a query, Neural Query Engine learns how to perform query planning and execution in order to efficiently retrieve the correct results by interacting with the Neural Graph Storage. Compared with traditional graph DBs, NGDBs allow for a flexible and unified modeling of features in diverse modalities using the embedding store. Moreover, when the graph is incomplete, they can provide robust retrieval of answers which a normal graph DB cannot recover. Finally, we point out promising directions, unsolved problems and applications of NGDB for future research.

Deep Sets

Abstract: We study the problem of designing models for machine learning tasks defined on sets. In contrast to the traditional approach of operating on fixed dimensional vectors, we consider objective functions defined on sets and are invariant to permutations. Such problems are widespread, ranging from the estimation of population statistics, to anomaly detection in piezometer data of embankment dams, to cosmology. Our main theorem characterizes the permutation invariant objective functions and provides a family of functions to which any permutation invariant objective function must belong. This family of functions has a special structure which enables us to design a deep network architecture that can operate on sets and which can be deployed on a variety of scenarios including both unsupervised and supervised learning tasks. We demonstrate the applicability of our method on population statistic estimation, point cloud classification, set expansion, and outlier detection.


For d = 5, 6 and r ≥ d − 2, Jin #REFR presented an explicit construction of q-ary optimal LRCs of length Ω r (q 2 ) via binary constant weight codes.

#OTHEREFR proposed a construction of cyclic optimal LRCs with unbounded length. For minimum distance d ≥ 5, Guruswami et al.

#OTHEREFR proved that the code length of a q-ary optimal LRC must be at most roughly O(dq 3 ) and presented an algorithmic construction of q-ary optimal LRCs of length

By employing algebraic structures of elliptic curves, Li et al.

#OTHEREFR constructed a family of q-ary optimal LRCs with length up to q + 2 √ q and some particular localities.

In #OTHEREFR , Xing and Yuan generalized Jin's results and presented a construction of q-ary optimal LRCs for general d ≥ 7 via hypergraph theory. In #OTHEREFR , Chen et al. considered the case of r < d − 2.

In particular, for d = 6 and r = 2, the authors established an equivalent connection between the existence of Singleton-optimal LRCs and the existence of lines in finite projective plane with certain properties.

And they also provided some new constructions and improved bounds on Singleton-optimal LRCs.

However, as we have seen, for d ≥ 5, the longest code length of known q-ary cyclic Singleton-optimal LRCs is q + 1 until now.

So it is quite interesting to construct q-ary cyclic Singleton-optimal LRCs with length n > q + 1 and d ≥ 5.

Singleton-Optimal Lrcs And Perfect Lrcs Via Cyclic And Constacyclic Codes *

Abstract: Locally repairable codes (LRCs) have emerged as an important coding scheme in distributed storage systems (DSSs) with relatively low repair cost by accessing fewer non-failure nodes. Theoretical bounds and optimal constructions of LRCs have been widely investigated. Optimal LRCs via cyclic and constacyclic codes provide significant benefit of elegant algebraic structure and efficient encoding procedure. In this paper, we continue to consider the constructions of optimal LRCs via cyclic and constacyclic codes with long code length. Specifically, we first obtain two classes of q-ary cyclic Singletonoptimal (n, k, d = 6; r = 2)-LRCs with length n = 3(q +1) when 3 | (q −1) and q is even, and length n = 3 2 (q + 1) when 3 | (q − 1) and q ≡ 1(mod 4), respectively. To the best of our knowledge, this is the first construction of q-ary cyclic * This paper was presented in part at the 2021 IEEE International Symposium on Information Theory (ISIT) [11]

Explicit Construction Of Optimal Locally Recoverable Codes Of Distance 5 And 6 Via Binary Constant Weight Codes

Abstract: In a paper by Guruswami et al. , it was shown that the length $n$ of a $q$ -ary linear locally recoverable code with distance $d \geqslant 5$ is upper bounded by $O(dq^{3})$ . Thus, it is a challenging problem to construct $q$ -ary locally recoverable codes with distance $d \geqslant 5$ and length approaching the upper bound. The same paper also gave an algorithmic construction of $q$ -ary locally recoverable codes with locality $r$ and length $n=\Omega _{r}(q^{2})$ for $d=5$ and 6, where $\Omega _{r}$ means that the implicit constant depends on locality $r$ . In this paper, we present an explicit construction of $q$ -ary locally recoverable codes of distance $d= 5$ and 6 via binary constant weight codes. It turns out that 1) our construction is simpler and more explicit and 2) the length of our codes is greater than previously known.


Following prior work #REFR , the MEAN aggregation function is used at each layer, and the dropout is set to 0.5 to prevent overfitting. Similarity Model.

The neighborhood sample size is fixed at ten samples at each layer.

We use a three-layer GAT model with a fixed neighborhood sample size of 10 at each layer.

The first and second layers have four attention heads each. GraphSAGE. We use a two-layer GraphSAGE model.

The neighborhood sample sizes are set to 25 and 10, respectively.

We use the Scikit-Learn implementation of an MLPClassifier to train C .

We train three versions of F for each dataset using the three architectures. Each F has its own C .

We train three versions of F and two versions of F for each F which we use to generate the positive and negative data points as explained in Section 5.

We use a two-layer MLP and find the best hyperparameters using a grid search.

The hidden layer sizes in the search are either 64 or 128, and the activation function is either Tanh or ReLU. We select the best model based on 10-fold crossvalidation.

Grove: Ownership Verification Of Graph Neural Networks Using Embeddings

Abstract: Graph neural networks (GNNs) have emerged as a state-of-the-art approach to model and draw inferences from large scale graphstructured data in various application settings such as social networking. The primary goal of a GNN is to learn an embedding for each graph node in a dataset that encodes both the node features and the local graph structure around the node. Embeddings generated by a GNN for a graph node are unique to that GNN. Prior work has shown that GNNs are prone to model extraction attacks. Model extraction attacks and defenses have been explored extensively in other non-graph settings. While detecting or preventing model extraction appears to be difficult, deterring them via effective ownership verification techniques offer a potential defense. In non-graph settings, fingerprinting models, or the data used to build them, have shown to be a promising approach toward ownership verification. We present GrOVe, a state-of-the-art GNN model fingerprinting scheme that, given a target model and a suspect model, can reliably determine if the suspect model was trained independently of the target model or if it is a surrogate of the target model obtained via model extraction. We show that GrOVe can distinguish between surrogate and independent models even when the independent model uses the same training dataset and architecture as the original target model. Using six benchmark datasets and three model architectures, we show that GrOVe consistently achieves low false-positive and falsenegative rates. We demonstrate that GrOVe is robust against known fingerprint evasion techniques while remaining computationally efficient.

Inductive Representation Learning On Large Graphs

Abstract: Low-dimensional embeddings of nodes in large graphs have proved extremely useful in a variety of prediction tasks, from content recommendation to identifying protein functions. However, most existing approaches require that all nodes in the graph are present during training of the embeddings; these previous approaches are inherently transductive and do not naturally generalize to unseen nodes. Here we present GraphSAGE, a general, inductive framework that leverages node feature information (e.g., text attributes) to efficiently generate node embeddings for previously unseen data. Instead of training individual embeddings for each node, we learn a function that generates embeddings by sampling and aggregating features from a node's local neighborhood. Our algorithm outperforms strong baselines on three inductive node-classification benchmarks: we classify the category of unseen nodes in evolving information graphs based on citation and Reddit post data, and we show that our algorithm generalizes to completely unseen graphs using a multi-graph dataset of protein-protein interactions.


Deep Adaptive Networks (DAN) #REFR uses a Hilbert space representation for the embedding representations to match different domain distributions.

For this reason, many synthetic datasets have been introduced in recent years including VisDA #OTHEREFR for domain adaptation techniques in image classification and segmentation, synthetic datasets for ground robots #OTHEREFR , #OTHEREFR , #OTHEREFR , aerial robots #OTHEREFR and action recognition including #OTHEREFR , #OTHEREFR , #OTHEREFR .

Action recognition from aerial videos has also been studied extensively #OTHEREFR and aided by datasets like #OTHEREFR , #OTHEREFR , #OTHEREFR . #OTHEREFR introduces the NEC-DRONE dataset for groundto-air domain adaptation.

Further, many simulators have also been developed to support generation of synthetic data.

These simulators, including CARLA #OTHEREFR , GTA #OTHEREFR , NVIDIA ISAAC/Omniverse #OTHEREFR , Habitat #OTHEREFR , #OTHEREFR , AI2Thor #OTHEREFR , and iGibson #OTHEREFR allow users to create virtual environments that provide error-free, ground truth annotations.

b) Synthetic-to-Real Transfer: Many studies have also explored synthetic-to-real transfer by leveraging synthetic data generated from simulation environments to pretrain a neural network, thereby reducing the amount of data needed in real-world settings #OTHEREFR , [38] .

Adversarial Discriminative Domain Adaptation (ADDA) #OTHEREFR and Generate to Adapt #OTHEREFR are domain adaptation techniques that use unsupervised data to learn a joint feature space between the source and target distributions.

CyCADA #OTHEREFR extends existing adversarial domain adaptation techniques by adding a cycle-consistency constraint for unsupervised adaptation from synthetic to real-world driving domains.

Randomized-to-Canonical Adaptation Networks (RCANs) #OTHEREFR , GraspGAN #OTHEREFR , RetinaGAN #OTHEREFR , Closing the Sim-to-Real Loop #OTHEREFR , and the work described in #OTHEREFR leverage simulation and domain adaptation techniques for robot manipulation and grasping.

VR-Goggles for Robots #OTHEREFR focuses on converting real images back to simulation for visual control of robots in indoor and outdoor environments.

The authors of #OTHEREFR use synthetic-toreal techniques to improve mobile robot control policies on uneven, complex outdoor environments.

Synthetic-To-Real Domain Adaptation For Action Recognition: A Dataset And Baseline Performances

Abstract: Human action recognition is a challenging problem, particularly when there is high variability in factors such as subject appearance, backgrounds and viewpoint. While deep neural networks (DNNs) have been shown to perform well on action recognition tasks, they typically require large amounts of high-quality labeled data to achieve robust performance across a variety of conditions. Synthetic data has shown promise as a way to avoid the substantial costs and potential ethical concerns associated with collecting and labeling enormous amounts of data in the real-world. However, synthetic data may differ from real data in important ways. This phenomenon, known as domain shift, can limit the utility of synthetic data in robotics applications. To mitigate the effects of domain shift, substantial effort is being dedicated to the development of domain adaptation (DA) techniques. Yet, much remains to be understood about how best to develop these techniques. In this paper, we introduce a new dataset called Robot Control Gestures (RoCoG-v2). The dataset is composed of both real and synthetic videos from seven gesture classes, and is intended to support the study of synthetic-to-real domain shift for videobased action recognition. Our work expands upon existing datasets by focusing the action classes on gestures for humanrobot teaming, as well as by enabling investigation of domain shift in both ground and aerial views. We present baseline results using state-of-the-art action recognition and domain adaptation algorithms and offer initial insight on tackling the synthetic-to-real and ground-to-air domain shifts. Instructions on accessing the dataset can be found at https://github. com/reddyav1/RoCoG-v2.

Learning Transferable Features With Deep Adaptation Networks

Abstract: Recent studies reveal that a deep neural network can learn transferable features which generalize well to novel tasks for domain adaptation. However, as deep features eventually transition from general to specific along the network, the feature transferability drops significantly in higher layers with increasing domain discrepancy. Hence, it is important to formally reduce the dataset bias and enhance the transferability in task-specific layers. In this paper, we propose a new Deep Adaptation Network (DAN) architecture, which generalizes deep convolutional neural network to the domain adaptation scenario. In DAN, hidden representations of all task-specific layers are embedded in a reproducing kernel Hilbert space where the mean embeddings of different domain distributions can be explicitly matched. The domain discrepancy is further reduced using an optimal multikernel selection method for mean embedding matching. DAN can learn transferable features with statistical guarantees, and can scale linearly by unbiased estimate of kernel embedding. Extensive empirical evidence shows that the proposed architecture yields state-of-the-art image classification error rates on standard domain adaptation benchmarks.


Previous research has shown that PD targets are an effective action parametrization for learning locomotion #REFR .

Figure 2 shows the height sampling taking place in the simulation environment.

Finally, = (n , n ) is the pair of noisy exteroceptive observations.

Sampling is similar to o however a noise is applied to the sampling pattern coordinates and sampled values.

Further details on noise generation are discussed in section II-E.

The action a ∈ R 10 represents the PD targets for the actuators in the robot model.

The robot PD controller runs at 2 kHz, whereas actions are sampled from the policy at a rate of 40 Hz.

Learning Perceptive Bipedal Locomotion Over Irregular Terrain

Abstract: In this paper we propose a novel bipedal locomotion controller that uses noisy exteroception to traverse a wide variety of terrains. Building on the cutting-edge advancements in attention based belief encoding for quadrupedal locomotion, our work extends these methods to the bipedal domain, resulting in a robust and reliable internal belief of the terrain ahead despite noisy sensor inputs. Additionally, we present a reward function that allows the controller to successfully traverse irregular terrain. We compare our method with a proprioceptive baseline and show that our method is able to traverse a wide variety of terrains and greatly outperforms the state-of-the-art in terms of robustness, speed and efficiency.

Learning Locomotion Skills Using Deeprl: Does The Choice Of Action Space Matter?

Abstract: The use of deep reinforcement learning allows for high-dimensional state descriptors, but little is known about how the choice of action representation impacts learning and the resulting performance. We compare the impact of four different action parameterizations (torques, muscle-activations, target joint angles, and target joint-angle velocities) in terms of learning time, policy robustness, motion quality, and policy query rates. Our results are evaluated on a gait-cycle imitation task for multiple planar articulated figures and multiple gaits. We demonstrate that the local feedback provided by higher-level action parameterizations can significantly impact the learning, robustness, and motion quality of the resulting policies.

locomotion
effective action parametrization


Specifically, the shallow feature extraction head consists of a stack of residual layers #REFR and 4× downsamples the features.

4.1, image dehazing is decoupled into two sub-tasks: matching correct code and removing texture distortion. Encoder for HQPs Matching.

We follow SwinIR #OTHEREFR which shows its powerful feature extraction ability for image restoration to design our encoder E.

Then 4 residual swin transformer blocks #OTHEREFR are followed, which serve as the deep feature extraction module. Decoder with Normalized Feature Alignment.

We propose the Normalized Feature Alignment (NFA) to help the decoder utilize the features reconstructed from HQPs.

Firstly, VQGAN tends to decrease results' fidelity due to the information loss brought by vector-quantized opera-

Ridcp: Revitalizing Real Image Dehazing Via High-Quality Codebook Priors

Abstract: Existing dehazing approaches struggle to process realworld hazy images owing to the lack of paired real data and robust priors. In this work, we present a new paradigm for real image dehazing from the perspectives of synthesizing more realistic hazy data and introducing more robust priors into the network. Specifically, (1) instead of adopting the de facto physical scattering model, we rethink the degradation of real hazy images and propose a phenomenological pipeline considering diverse degradation types. (2) We propose a Real Image Dehazing network via high-quality Codebook Priors (RIDCP). Firstly, a VQGAN is pre-trained on a large-scale high-quality dataset to obtain the discrete codebook, encapsulating high-quality priors (HQPs). After replacing the negative effects brought by haze with HQPs, the decoder equipped with a novel normalized feature alignment module can effectively utilize high-quality features and produce clean results. However, although our degradation pipeline drastically mitigates the domain gap between synthetic and real data, it is still intractable to avoid it, which challenges HQPs matching in the wild. Thus, we recalculate the distance when matching the features to the HQPs by a controllable matching operation, which facilitates finding better counterparts. We provide a recommendation to control the matching based on an explainable solution. Users can also flexibly adjust the enhancement degree as per their preference. Extensive experiments verify the effectiveness of our data synthesis pipeline and the superior performance of RIDCP in real image dehazing.

Deep Residual Learning For Image Recognition

Abstract: Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.


Furthermore, in these complex settings systems often lack the sort of commonsense knowledge that is required to effectively operate in a human-centered environment #REFR .

However, prescribing that disclosure takes place doesn't ensure that the disclosure contains useful information, or the information provided will be relevant and understandable to those consuming it.

To address this issue, work on qualitatively evaluate the disclosure sufficiency with rubrics #OTHEREFR , or automatically assessing the layperson-comprehensibility of a system function description have been proposed.

A further challenge to system function disclosure is that, in many cases, even experts don't know precisely how black box systems mechanistically produce output from input #OTHEREFR -thus explainability and interpretability techniques can be prerequisites for the level of expert understanding needed to produce honest disclosure ( §3.2).

A common limitation to many ML techniques applied in sensitive settings (e.g., medicine, criminal justice, employment) is the invisibility of external social context to the model.

As biases and oversights in training data propagate to learned systems, system developers require clarity from data providers ( §2.1) to ensure that they can in turn communicate the problems of their systems #OTHEREFR .

Apart from solving the problem of commonsense reasoning in AI, actively soliciting direct user feedback to contextualize failure cases #OTHEREFR is one practical way to bootstrap documentation of model weaknesses.

However, even if perfect information about a model's strengths and weaknesses (which is difficult to gain) and strong explanations of its internal functional details are available, calibrating explanations to be comprehensible to diverse stakeholders is still a confounding problem.

In short, different groups require different explanations, and have different levels of expertise.

Insufficient disclosure may be unsatisfy-ing, but too much disclosure may result in information overload, and lower user trust #OTHEREFR .

To combat this issue, it is crucial to understand the desired ends of each stakeholder and information in a manner that balances this duality [Vaughan and Wallach, 2020].

Users Are The North Star For Ai Transparency

Abstract: Despite widespread calls for transparent artificial intelligence systems, the term is too overburdened with disparate meanings to express precise policy aims or to orient concrete lines of research. Consequently, stakeholders often talk past each other, with policymakers expressing vague demands and practitioners devising solutions that may not address the underlying concerns. Part of why this happens is that a clear ideal of AI transparency goes unsaid in this body of work. We explicitly name such a north star-transparency that is usercentered, user-appropriate, and honest. We conduct a broad literature survey, identifying many clusters of similar conceptions of transparency, tying each back to our north star with analysis of how it furthers or hinders our ideal AI transparency goals. We conclude with a discussion on common threads across all the clusters, to provide clearer common language whereby policymakers, stakeholders, and practitioners can communicate concrete demands and deliver appropriate solutions. We hope for future work on AI transparency that further advances confident, user-beneficial goals and provides clarity to regulators and developers alike. Perspective Definition of transparency Public Policy Any meaningful information relating to consumer data is disclosed in comprehensible language [Voigt, 2018; on AI, 2019]. Data Collection Disclosure of collection methods and privacy policies in a consumer-understandable manner [Driscoll and Walker, 2014; Agozie and Kaya, 2021]. Data Processing Comprehensible disclosure of methods in which consumer data is processed, stored, and used [Kirrane et al., 2021]. Reproducibility Disclosure of important information to reproduce a system's performance [Gundersen and Kjensmo, 2018] Intelligibility Disclosure of pertinent system functionality and limitations comprehensible to stakeholders [Vaughan and Wallach, 2020; Ehsan et al., 2021]. Interpretability Explanation that aids understanding of system functionality [Lipton, 2018; Watson and Nations, 2019]. Fairness Disclosure regarding representation and treatment to ensure equity among groups [Castillo, 2019; Bhatt et al., 2021].

Automated Rationale Generation: A Technique For Explainable Ai And Its Effects On Human Perceptions

Abstract: Automated rationale generation is an approach for real-time explanation generation whereby a computational model learns to translate an autonomous agent's internal state and action data representations into natural language. Training on human explanation data can enable agents to learn to generate human-like explanations for their behavior. In this paper, using the context of an agent that plays Frogger, we describe (a) how to collect a corpus of explanations, (b) how to train a neural rationale generator to produce different styles of rationales, and (c) how people perceive these rationales. We conducted two user studies. The first study establishes the plausibility of each type of generated rationale and situates their user perceptions along the dimensions of confidence, humanlike-ness, adequate justification, and understandability. The second study further explores user preferences between the generated rationales with regard to confidence in the autonomous agent, communicating failure and unexpected behavior. Overall, we find alignment between the intended differences in features of the generated rationales and the perceived differences by users. Moreover, context permitting, participants preferred detailed rationales to form a stable mental model of the agent's behavior.


It is worth noting that earlier work has shown similar numbers to the ones we report for KR, but for other models and evaluation sets, such as BERT with LAMA #REFR .

We note that KR as memorization could be considered a privacy breach, hence we consider the presence PII as one of our discourse flaws.

We direct the reader to the survey by Liu et al.

#OTHEREFR on prompting methods; the work by Zhou et al.

#OTHEREFR on using LLMs to generate prompts; and existing benchmarks such as Piqa #OTHEREFR , LAMA #OTHEREFR , and BIG-Bench-Hard #OTHEREFR for evaluating the performance of LLMs across multiple tasks.

Our work is not designed to be an automated benchmark, and instead focuses on manual analysis of the text.

#OTHEREFR performed an authoritative evaluation on LLMs prior to the release of some of the models evaluated, to which we point the interested reader.

Our work is narrower in scope, and focuses on the quality of the output as seen by an user without training, and by the easier-to-access models.

Our KR findings are largely in line with theirs, although we find the incidence of output of memorized content to be higher.

We also expand on the correlation between memorization and accuracy, by noting that this extends to overall discourse quality.

An Evaluation On Large Language Model Outputs: Discourse And Memorization

Abstract: We present an empirical evaluation of various outputs generated by nine of the most widely-available large language models (LLMs). Our analysis is done with off-the-shelf, readily-available tools. We find a correlation between percentage of memorized text, percentage of unique text, and overall output quality, when measured with respect to output pathologies such as counterfactual and logically-flawed statements, and general failures like not staying on topic. Overall, 80.0% of the outputs evaluated contained memorized data, but outputs containing the most memorized content were also more likely to be considered of high quality. We discuss and evaluate mitigation strategies, showing that, in the models evaluated, the rate of memorized text being output is reduced. We conclude with a discussion on potential implications around what it means to learn, to memorize, and to evaluate quality text.

How Can We Know What Language Models Know?

Abstract: Recent work has presented intriguing results examining the knowledge contained in language models (LM) by having the LM fill in the blanks of prompts such as "Obama is a _ by profession". These prompts are usually manually created, and quite possibly sub-optimal; another prompt such as "Obama worked as a _" may result in more accurately predicting the correct profession. Because of this, given an inappropriate prompt, we might fail to retrieve facts that the LM does know, and thus any given prompt only provides a lower bound estimate of the knowledge contained in an LM. In this paper, we attempt to more accurately estimate the knowledge contained in LMs by automatically discovering better prompts to use in this querying process. Specifically, we propose mining-based and paraphrasing-based methods to automatically generate high-quality and diverse prompts and ensemble methods to combine answers from different prompts. Extensive experiments on the LAMA benchmark for extracting relational knowledge from LMs demonstrate that our methods can improve accuracy from 31.1% to 38.1%, providing a tighter lower bound on what LMs know. We have released the code and the resulting LM Prompt And Query Archive (LPAQA) at this https URL.


C. Robust Data-Based Controller Using Noisy Data

System (1) with controller (32) can be described by the composite system #REFR , presented at the top of this page.

Ξ 3 = (X 1 − W 0 )Φ † 1 + (X 1 − W 0 )Φ † 2K (X 1 − W 0 )Φ † 2K 0 X 1Φ † 1 −LY 0Φ † 1 Ξ1 + 0 0 W 0Φ † 1 +W 0Φ † 2K −LV 0Φ † 1 W 0Φ † 2K Ξ2 (35)

Considering the estimation error e(t) = x(t)−x(t), system (33) can be transformed into the composite system #OTHEREFR .

Since the transformation from [x(t) x (t)] to [x(t) e (t)] is linear and non-singular, matrix Ξ 3 is similar to Ξ 0 .

From #OTHEREFR Section 6.6] , this implies that system (33) has same characteristic polynomial as system #OTHEREFR .

Hence, if Ξ 3 is Schur stable, so is Ξ 0 . According to #OTHEREFR , one has that

Learning Robust Data-Based Lqg Controllers From Noisy Data

Abstract: This paper addresses the joint state estimation and control problems for unknown linear time-invariant systems subject to both process and measurement noise. The aim is to redesign the linear quadratic Gaussian (LQG) controller based solely on data. The LQG controller comprises a linear quadratic regulator (LQR) and a steady-state Kalman observer; while the data-based LQR design problem has been previously studied, constructing the Kalman gain and the LQG controller from noisy data presents a novel challenge. In this work, a data-based formulation for computing the steady-state Kalman gain is proposed based on semi-definite programming (SDP) using some noise-free input-state-output data. Additionally, a data-based LQG controller is developed, which is shown to be equivalent to the model-based LQG controller. For cases where offline data are corrupted by noise, a robust data-based observer gain is constructed by tackling a relaxed SDP. The proposed controllers are proven to achieve robust global exponential stability (RGES) for state estimation and input-to-state practical stability (ISpS) under standard conditions. Finally, numerical tests are conducted to validate the proposed controllers' correctness and effectiveness. Index terms-Data-driven control, state estimation, noisy data, semi-definite program, linear quadratic Gaussian. I. INTRODUCTION The states of a dynamical system refer to a set of variables that provide a complete characterization of its internal conditions or status at a given time [35], which plays a fundamental role in control, monitoring, and high-level information extraction of real-world systems. Unfortunately, the complete states are not always fully observable; and even if measurements can be collected, the inevitable sensor miscalibration and noise may degrade the accuracy. As such, using the most likely value of a state reconstructed from raw measurements serves as an alternative, that is what is known as state estimation. Numerous efforts have contributed to the development of state estimation methods and theory as well as their applications in networked control systems, cyber-physical systems, and smart power grids; see e.g., [9], [10], [33], [39]. Most existing results on state estimation capitalize on explicit system models describing the underlying dynamics. However,

Distributed Estimation Via Iterative Projections With Application To Power Network Monitoring

Abstract: This work presents a distributed method for control centers to monitor the operating condition of a power network, i.e., to estimate the network state, and to ultimately determine the occurrence of threatening situations. State estimation has been recognized to be a fundamental task for network control centers to operate safely and reliably a power grid. We consider (static) state estimation problems, in which the state vector consists of the voltage magnitude and angle at all network buses. We consider the state to be linearly related to network measurements, which include power flows, current injections, and voltage phasors at some buses. We admit the presence of several cooperating control centers, and we design two distributed methods for them to compute the minimum variance estimate of the state, given the network measurements. The two distributed methods rely on different modes of cooperation among control centers: in the first method an incremental mode of cooperation is used, whereas, in the second method, a diffusive interaction is implemented. Our procedures, which require each control center to know only the measurements and the structure of a subpart of the whole network, are computationally efficient and scalable with respect to the network dimension, provided that the number of control centers also increases with the network cardinality. Additionally, a finite-memory approximation of our diffusive algorithm is proposed, and its accuracy is characterized. Finally, our estimation methods are exploited to develop a distributed algorithm to detect corrupted network measurements.


Debiased Recommendation With A Randomized Dataset

Propensity score recommendation learning is a representative work in this sub-route, and proposes two methods for the propensity score estimation based on a randomized dataset, including a naive Bayes estimator and a regression model estimator #REFR .

The research on this route additionally introduces a randomized dataset that can act as a proxy for the unbiased information.

Most debiasing methods that fall into this route aim to mine the unbiased knowledge from a randomized dataset by formulating some more sophisticated and efficient techniques, and then use them to guide the training process of a recommendation model on a non-randomized dataset.

The existing works on this research route can be further subdivided into three classes, including inverse propensity score and imputation labels-based methods, multi-stage training-based methods, and joint training-based methods, depending on the different techniques employed.

An inverse propensity score and imputation labels-based method utilizes an additional randomized dataset to estimate the propensity score for each feedback #OTHEREFR or to make the predictions of the imputation labels for unobserved feedback data #OTHEREFR .

These obtained propensity scores or imputation labels will be integrated into the model's optimization objective, i.e., transfer the unbiased knowledge into the model's training process.

Note that the propensity scores are used in both debiased recommendation routes, and they differ in whether the propensity score is estimated from a non-randomized dataset or a randomized dataset.

In addition, some works also consider estimating and using the propensity scores and imputation labels simultaneously to allow the model to benefit more in a doubly robust framework #OTHEREFR .

A multi-stage training-based method designs some effective multi-stage training frameworks in which a non-randomized dataset and a randomized data are used alternately, based on the synergy of which it learns better unbiased parameters.

AutoDebias #OTHEREFR is one of the most representative methods on this research sub-line.

Its main idea is to introduce a meta-learning strategy into a doubly robust debiasing framework to achieve better learning of the model.

Bounding System-Induced Biases In Recommender Systems With A Randomized Dataset

Abstract: Debiased recommendation with a randomized dataset has shown very promising results in mitigating the system-induced biases. However, it still lacks more theoretical insights or an ideal optimization objective function compared with the other more well studied route without a randomized dataset. To bridge this gap, we study the debiasing problem from a new perspective and propose to directly minimize the upper bound of an ideal objective function, which facilitates a better potential solution to the system-induced biases. Firstly, we formulate a new ideal optimization objective function with a randomized dataset. Secondly, according to the prior constraints that an adopted loss function may satisfy, we derive two different upper bounds of the objective function, i.e., a generalization error bound with the triangle inequality and a generalization error bound with the separability. Thirdly, we show that most existing related methods can be regarded as the insufficient optimization of these two upper bounds. Fourthly, we propose a novel method called debiasing approximate upper bound with a randomized dataset (DUB), which achieves a more sufficient optimization of these upper bounds. Finally, we conduct extensive experiments on a public dataset and a real product dataset to verify the effectiveness of our DUB.

Recommendations As Treatments: Debiasing Learning And Evaluation

Abstract: Most data for evaluating and training recommender systems is subject to selection biases, either through self-selection by the users or through the actions of the recommendation system itself. In this paper, we provide a principled approach to handle selection biases by adapting models and estimation techniques from causal inference. The approach leads to unbiased performance estimators despite biased data, and to a matrix factorization method that provides substantially improved prediction performance on real-world data. We theoretically and empirically characterize the robustness of the approach, and find that it is highly practical and scalable.


Our initial finding suggests that not all input channels need to undergo the computationally expensive depthwise convolution operation #REFR , as demonstrated by our preliminary experiment results (see Table 10 ).

As shown in Figure 1 , the ConvNeXt-T with a default 7 × 7 kernel size is 1.4× slower than that with small kernel size of 3 × 3, and is 1.8× slower than ResNet-50, although they have similar FLOPs.

However, using a smaller kernel size limits the receptive field, which can result in performance decrease.

For example, ConvNeXt-T/k3 suffers a performance drop of 0.6% top-1 accuracy on the ImageNet-1K dataset when compared to ConvNeXt-T/k7 (where kn denotes a kernel size of n × n).

It is still unclear how to speed up large-kernel CNNs while preserving their performance.

In this paper, we aim to address this issue by building upon ConvNeXt as our baseline and improving the depthwise convolution module.

Therefore, we propose to leave some channels unaltered and process only a portion of the channels with the depthwise convolution operation.

Next, we propose to decompose large kernel of depthwise convolution into several groups of small kernels with Inception style #OTHEREFR .

Specifically, for the processing channels, 1/3 of channels are conducted with kernel of 3 × 3, 1/3 are with 1 × k while the remaining 1/3 are with k × 1.

With this new simple and cheap operator, termed as "Inception depthwise convolution", our built model Inception-NeXt achieves a much better trade-off between accuracy and speed.

For example, as shown in Figure 1 , InceptionNeXt-T achieves higher accuracy than ConvNeXt-T while enjoying 1.6× speedup of training throughput similar to ResNet-50.

Inceptionnext: When Inception Meets Convnext

Abstract: Inspired by the long-range modeling ability of ViTs, large-kernel convolutions are widely studied and adopted recently to enlarge the receptive field and improve model performance, like the remarkable work ConvNeXt which employs 7 × 7 depthwise convolution. Although such depthwise operator only consumes a few FLOPs, it largely harms the model efficiency on powerful computing devices due to the high memory access costs. For example, ConvNeXt-T has similar FLOPs with ResNet-50 but only achieves 60% throughputs when trained on A100 GPUs with full precision. Although reducing the kernel size of ConvNeXt can improve speed, it results in significant performance degradation. It is still unclear how to speed up large-kernelbased CNN models while preserving their performance. To tackle this issue, inspired by Inceptions, we propose to decompose large-kernel depthwise convolution into four parallel branches along channel dimension, i.e. small square kernel, two orthogonal band kernels, and an identity mapping. With this new Inception depthwise convolution, we build a series of networks, namely IncepitonNeXt, which not only enjoy high throughputs but also maintain competitive performance. For instance, InceptionNeXt-T achieves 1.6× higher training throughputs than ConvNeX-T, as well as attains 0.2% top-1 accuracy improvement on ImageNet-1K. We anticipate InceptionNeXt can serve as an economical baseline for future architecture design to reduce carbon footprint.

Shufflenet V2: Practical Guidelines For Efficient Cnn Architecture Design

Abstract: Currently, the neural network architecture design is mostly guided by the \emph{indirect} metric of computation complexity, i.e., FLOPs. However, the \emph{direct} metric, e.g., speed, also depends on the other factors such as memory access cost and platform characterics. Thus, this work proposes to evaluate the direct metric on the target platform, beyond only considering FLOPs. Based on a series of controlled experiments, this work derives several practical \emph{guidelines} for efficient network design. Accordingly, a new architecture is presented, called \emph{ShuffleNet V2}. Comprehensive ablation experiments verify that our model is the state-of-the-art in terms of speed and accuracy tradeoff.


For many applications in the noiseless setting, simple ℓ 1 -minimization has proven to be the approach of choice due to its relative conceptual ease, implementability as a linear program, and favorable performance even when compared with tailored regularizers #REFR .

A linear system is, in particular, a compact means of describing a linear relationship y = Ψu between a sequence of of inputs u and a sequence of observations y, from which even in the most optimistic circumstances generic u can only be reconstructed up to ker Ψ.

However, by assuming that u is sparse in an appropriate sense, established results in sparse recovery provide favorable guarantees on exact reconstruction for an appropriate choice of optimization algorithm.

The most common such case considered is when u is assumed to have support of size not greater than s, and is termed regular sparsity.

Other support-based notions of sparsity include block #OTHEREFR , group #OTHEREFR , and tree-based sparsity #OTHEREFR , and are each subsumed by the more general notion of model-based sparsity #OTHEREFR .

For each of these sparsity patterns, the literature has provided tailored optimization problems and recovery guarantees, with varying levels of robustness to noise, ease of checking, and conceptual nuance.

For regular sparsity, the necessary and sufficient condition for successful unique recovery is the satisfaction of the so-called nullspace property (NUP), which requires vectors in the nullspace of Ψ to have smaller ℓ 1 norm on s-sparse supports than on the complement of s-sparse supports (see Def. 1).

Recent results have even shown that for any support-based notion of sparsity, there exists a straightforward extension of the NUP, termed the generalized nullspace property, which provides necessary and sufficient guarantees #OTHEREFR .

In light of the success of this approach to signal reconstruction, recent literature has provided tailored algorithms for sparse recovery in linear dynamical systems (LDSs), where the assumption of sparsity has been variously made on the initial conditions #OTHEREFR , #OTHEREFR , #OTHEREFR , #OTHEREFR , dynamics #OTHEREFR , measurement noise #OTHEREFR , and inputs #OTHEREFR , #OTHEREFR , #OTHEREFR , #OTHEREFR .

Even with all of this prior work, there are few existing guarantees on the performance of these algorithms, and the guarantees that have been produced are typically probabilistic in nature or make restrictive assumptions on the sparsity patterns, such as the state and inputs being simultaneously sparse with respect to an orthogonal dictionary.

As a result, many results for the general, noiseless setting, including the establishment of necessary and sufficient guarantees, have not yet appeared in the literature.

Necessary And Sufficient Conditions For Simultaneous State And Input Recovery Of Linear Systems With Sparse Inputs By ℓ 1 -Minimization

Abstract: The study of theoretical conditions for recovering sparse signals from compressive measurements has received a lot of attention in the research community. In parallel, there has been a great amount of work characterizing conditions for the recovery both the state and the input to a linear dynamical system (LDS), including a handful of results on recovering sparse inputs. However, existing sufficient conditions for recovering sparse inputs to an LDS are conservative and hard to interpret, while necessary and sufficient conditions have not yet appeared in the literature. In this work, we provide (1) the first characterization of necessary and sufficient conditions for the existence and uniqueness of sparse inputs to an LDS, (2) the first necessary and sufficient conditions for a linear program to recover both an unknown initial state and a sparse input, and (3) simple, interpretable recovery conditions in terms of the LDS parameters. We conclude with a numerical validation of these claims and discuss implications and future directions.

Block-Sparse Recovery Via Convex Optimization

Abstract: Given a dictionary that consists of multiple blocks and a signal that lives in the range space of only a few blocks, we study the problem of finding a block-sparse representation of the signal, i.e., a representation that uses the minimum number of blocks. Motivated by signal/image processing and computer vision applications, such as face recognition, we consider the block-sparse recovery problem in the case where the number of atoms in each block is arbitrary, possibly much larger than the dimension of the underlying subspace. To find a block-sparse representation of a signal, we propose two classes of non-convex optimization programs, which aim to minimize the number of nonzero coefficient blocks and the number of nonzero reconstructed vectors from the blocks, respectively. Since both classes of problems are NP-hard, we propose convex relaxations and derive conditions under which each class of the convex programs is equivalent to the original non-convex formulation. Our conditions depend on the notions of mutual and cumulative subspace coherence of a dictionary, which are natural generalizations of existing notions of mutual and cumulative coherence. We evaluate the performance of the proposed convex programs through simulations as well as real experiments on face recognition. We show that treating the face recognition problem as a block-sparse recovery problem improves the state-of-the-art results by 10% with only 25% of the training data.


We follow the experimental setup of PointNetLK #REFR by dividing the dataset into two parts: 20 classes for training, and another 20 classes for cross-class testing.

We train and test our method on the Model-Net40 #OTHEREFR dataset, which consists of 12311 CAD models from 40 object classes, containing 3D CAD models from 40 classes.

The source point cloud is the vertices from ModelNet40, which we uniformly sample 1024 points and normalize to a unit box at the origin [0, 1] 3 .

Following #OTHEREFR , the rotation along each axis is uniformly sampled in [0, 45 • ], and the translation is in [−0.5, 0.5].

The source point cloud is rigidly transformed to generate the target point cloud, which is then used to train and evaluate the registration model.

3DMatch: 3DMatch #OTHEREFR is a collection of several real-world indoor datasets and a widely used registration benchmark that captures data using Kinect cameras, including complex 3D RGB-D scans obtained from multiple scenes such as offices, hotels, and kitchens.

Following PointNetLK-Revisited #OTHEREFR , we chose 8 categories of scenes from 7-Scenes #OTHEREFR and SUN3D #OTHEREFR for testing, and the rest are used for training.

Sdfreg: Learning Signed Distance Functions For Point Cloud Registration

Abstract: Learning-based point cloud registration methods can handle clean point clouds well, while it is still challenging to generalize to noisy and partial point clouds. To this end, we propose a novel framework for noisy and partial point cloud registration. By introducing a neural implicit function representation, we replace the problem of rigid registration between point clouds with a registration problem between the point cloud and the neural implicit function. We then alternately optimize the implicit function representation and the registration between the implicit function and point cloud. In this way, point cloud registration can be performed in a coarse-to-fine manner. Since our method avoids computing point correspondences, it is robust to the noise and incompleteness of point clouds. Compared with the registration methods based on global features, our method can deal with surfaces with large density variations and achieve higher registration accuracy. Experimental results and comparisons demonstrate the effectiveness of the proposed framework.

Pointnetlk: Robust & Efficient Point Cloud Registration Using Pointnet

Abstract: PointNet has revolutionized how we think about representing point clouds. For classification and segmentation tasks, the approach and its subsequent extensions are state-of-the-art. To date, the successful application of PointNet to point cloud registration has remained elusive. In this paper we argue that PointNet itself can be thought of as a learnable "imaging" function. As a consequence, classical vision algorithms for image alignment can be applied on the problem - namely the Lucas & Kanade (LK) algorithm. Our central innovations stem from: (i) how to modify the LK algorithm to accommodate the PointNet imaging function, and (ii) unrolling PointNet and the LK algorithm into a single trainable recurrent deep neural network. We describe the architecture, and compare its performance against state-of-the-art in common registration scenarios. The architecture offers some remarkable properties including: generalization across shape categories and computational efficiency - opening up new paths of exploration for the application of deep learning to point cloud registration. Code and videos are available at this https URL.


Theorem 4.2 shows that the gauge map is differentiable with respect to the output of the neural layers, and hence enables the computation of backpropagation gradients in #REFR and the training of the architecture in Fig. 1 .

Here, we show that the non-differentiability introduced by the gauge map is no more severe than the non-differentiability that is already present in the neural network activation functions, and the end-to-end policy is differentiable almost everywhere: Theorem 4.2. Let P and Q be polyhedral C-sets.

Standard automatic differentiation procedures, when applied to the gauge map Gp¨| P, Qq, will return the gradient of Gp¨| P, Qq for almost all z P P.

The set P can be partitioned such that the gauge map is a different analytic function on each region of the partition (excluding the origin).

By setting Gp0 | P, Qq :" 0, we obtain a function for which standard automatic differentiation procedures will compute the gradient of Gp¨| P, Qq at all z P P except possibly on a set of measure zero #OTHEREFR . Details are in Appendix E.

The effectiveness of the proposed learning architecture is validated on a modified IEEE 118-bus system as shown in the next section.

An Efficient Learning-Based Solver For Two-Stage Dc Optimal Power Flow With Feasibility Guarantees

Abstract: In this paper, we consider the scenario-based twostage stochastic DC optimal power flow (OPF) problem for optimal and reliable dispatch when the load is facing uncertainty. Although this problem is a linear program, it remains computationally challenging to solve due to the large number of scenarios needed to accurately represent the uncertainties. To mitigate the computational issues, many techniques have been proposed to approximate the second-stage decisions so they can dealt more efficiently. The challenge of finding good policies to approximate the second-stage decisions is that these solutions need to be feasible, which has been difficult to achieve with existing policies. To address these challenges, this paper proposes a learning method to solve the two-stage problem in a more efficient and optimal way. A technique called the gauge map is incorporated into the learning architecture design to guarantee the learned solutions' feasibility to the network constraints. Namely, we can design policies that are feed forward functions that only output feasible solutions. Simulation results on standard IEEE systems show that, compared to iterative solvers and the widely used affine policy, our proposed method not only learns solutions of good quality but also accelerates the computation by orders of magnitude.

Corrective Control To Handle Forecast Uncertainty: A Chance Constrained Optimal Power Flow

Abstract: Higher shares of electricity generation from renewable energy sources and market liberalization is increasing uncertainty in power systems operation. At the same time, operation is becoming more flexible with improved control systems and new technology such as phase shifting transformers (PSTs) and high voltage direct current connections (HVDC). Previous studies have shown that the use of corrective control in response to outages contributes to a reduction in operating cost, while maintaining N-1 security. In this work, we propose a method to extend the use of corrective control of PSTs and HVDCs to react to uncertainty. We characterize the uncertainty as continuous random variables, and define the corrective control actions through affine control policies. This allows us to efficiently model control reactions to a large number of uncertainty sources. The control policies are then included in a chance constrained optimal power flow formulation, which guarantees that the system constraints are enforced with a desired probability. By applying an analytical reformulation of the chance constraints, we obtain a second-order cone problem for which we develop an efficient solution algorithm. In a case study for the IEEE 118 bus system, we show that corrective control for uncertainty leads to a decrease in operational cost, while maintaining system security. Further, we demonstrate the scalability of the method by solving the problem for the IEEE 300 bus and the Polish system test cases.


Another limitation is such tools' propensity to degenerate, that is, to generate bland and repetitive text #REFR .

Subsequent research could fine-tune such models to improve their capacity to generate appropriate language for the target text type and explore such models' contributions to student writing.

Besides, study should expand to the largest, proprietary language language models like ChatGPT and to AI-NLG tools that can generate several paragraphs of text.

Although AI-NLG tools hold promise to improve students' writing, SOTA AI-NLG tools that predict text suffer limitations and their use raises risk in the writing classroom.

One limitation is such tools' propensity to hallucinate, that is, to generate text that deviates from its source input and fails to meet user expectations #OTHEREFR .

Such text can comprise offensive or biased material #OTHEREFR , or factually incorrect and nonsensical answers, although plausible sounding #OTHEREFR .

Therefore, research will be necessary into prompt programming #OTHEREFR of SOTA AI-NLG tools in the writing classroom, that is, the careful selection of input by which a student interacts with and controls an AI-NLG tool to perform a desired task.

Since rewriting a prompt can significantly change a tool's performance, future studies should investigate the conditions under which a student would prompt a SOTA AI-NLG tool, what prompts students have used to generate text, and whether students perceive any AI-generated text as satisfactory for integration into a composition.

Future research may explore students' interactions with AI-NLG tools in the writing process, using methods such as screen recordings, think-aloud protocols, interviews, and stimulated recalls, to provide a complete understanding of how the tools contribute to students'

writing and how different students use AI for writing.

Exploring Ai-Generated Text In Student Writing: How Does Ai Help?

Abstract: English as a foreign language (EFL) students' use of text generated from artificial intelligence (AI) natural language generation (NLG) tools may improve their writing quality. However, it remains unclear to what extent AI-generated text in these students' writing might lead to higherquality writing. We explored 23 Hong Kong secondary school students' attempts to write stories comprising their own words and AI-generated text. Human experts scored the stories for dimensions of content, language, and organization. We analyzed the basic organization and structure and syntactic complexity of the stories' AI-generated text and performed multiple linear regression and cluster analyses. The results show the number of human words and the number of AI-generated words contribute significantly to scores. Besides, students can be grouped into competent and less competent writers who use more AI-generated text or less AI-generated text compared to their peers. Comparisons of the clusters reveal some benefit of AI-generated text in improving the quality of both high-scoring students' and low-scoring students' writing. The findings can inform pedagogical strategies to use AI-generated text for EFL students' writing and to address digital divides. This study contributes designs of NLG tools and writing activities to implement AI-generated text in schools.

The Curious Case Of Neural Text Degeneration

Abstract: Despite considerable advances in neural language modeling, it remains an open question what the best decoding strategy is for text generation from a language model (e.g. to generate a story). The counter-intuitive empirical observation is that even though the use of likelihood as training objective leads to high quality models for a broad range of language understanding tasks, maximization-based decoding methods such as beam search lead to degeneration — output text that is bland, incoherent, or gets stuck in repetitive loops. To address this we propose Nucleus Sampling, a simple but effective method to draw considerably higher quality text out of neural language models. Our approach avoids text degeneration by truncating the unreliable tail of the probability distribution, sampling from the dynamic nucleus of tokens containing the vast majority of the probability mass. To properly examine current maximization-based and stochastic decoding methods, we compare generations from each of these methods to the distribution of human text along several axes such as likelihood, diversity, and repetition. Our results show that (1) maximization is an inappropriate decoding objective for open-ended text generation, (2) the probability distributions of the best current language models have an unreliable tail which needs to be truncated during generation and (3) Nucleus Sampling is the best decoding strategy for generating long-form text that is both high-quality — as measured by human evaluation — and as diverse as human-written text.


We train a vanilla IMPALA #REFR agent for the initial data gathering in each environment.

In TreeMaze, every interaction, including opening a door, picking up a key, and interacting with other objects, is seen as an achievement.

The rooms are generated with the guarantee that it's possible to unlock all achievements.

The locations of the rooms are randomized but the dependency graph is fixed.

We use three versions of TreeMaze in our experiments, the TreeMaze-Easy which contains 4 rooms and 9 achievements, the TreeMaze-Normal which contains 10 rooms and 19 achievements and the TreeMaze-Hard that contains 18 rooms and 30 achievements.

For the Crafter environment, we also include a limited version (Crafter-Limited) which only contains 7 of the 21 achievements in the full environment.

In the full Crafter environment, the IMPALA agent discovers 17 achievements (with > 1% reaching rate); In all the remaining environments, the IMPALA agents discover all of the achievements.

We consider the 17 achievements reachable by a vanilla IM-PALA agent as the easy achievement set, and the remaining four achievements (make iron pickaxe, make iron sword, defeat zombie, collect diamond) as the hard achievement set.

The reason why these achievements are particularly hard to complete is that crafting iron tools in Crafter has three prerequisites: the character near a crafting table, the character near a furnace, the character has at least one wood, one coal, and one iron in the inventory.

Of the 4 hard achievements, collect diamond is objectively the hardest achievement to unlock as not only does it depends on the make iron pickaxe, but also that the diamond resource is extremely scarce in the game. The complete achievement list can be found in Sec. B.1.

Learning Achievement Structure For Struc-Tured Exploration In Domains With Sparse Re-Ward

Abstract: We propose Structured Exploration with Achievements (SEA), a multi-stage reinforcement learning algorithm designed for achievement-based environments, a particular type of environment with an internal achievement set. SEA first uses offline data to learn a representation of the known achievements with a determinant loss function, then recovers the dependency graph of the learned achievements with a heuristic algorithm, and finally interacts with the environment online to learn policies that master known achievements and explore new ones with a controller built with the recovered dependency graph. We empirically demonstrate that SEA can recover the achievement structure accurately and improve exploration in hard domains such as Crafter that are procedurally generated with highdimensional observations like images.

Impala: Scalable Distributed Deep-Rl With Importance Weighted Actor-Learner Architectures

Abstract: In this work we aim to solve a large collection of tasks using a single reinforcement learning agent with a single set of parameters. A key challenge is to handle the increased amount of data and extended training time, which is already a problem in single task learning. We have developed a new distributed agent IMPALA (Importance-Weighted Actor Learner Architecture) that can scale to thousands of machines and achieve a throughput rate of 250,000 frames per second. We achieve stable learning at high throughput by combining decoupled acting and learning with a novel off-policy correction method called V-trace, which was critical for achieving learning stability. We demonstrate the effectiveness of IMPALA for multi-task reinforcement learning on DMLab-30 (a set of 30 tasks from the DeepMind Lab environment (Beattie et al., 2016)) and Atari-57 (all available Atari games in Arcade Learning Environment (Bellemare et al., 2013a)). Our results show that IMPALA is able to achieve better performance than previous agents, use less data and crucially exhibits positive transfer between tasks as a result of its multi-task approach.

initial data gathering
vanilla IMPALA agent


As compared to ELAN, the results of the studies show that using DCELAN results in improvements in mDice and mIOU of 4.82% and 2.78%, respectively, on the CVC-ClinicDB #REFR dataset.

The following ablation experiments were carried out by us in order to evaluate the effectiveness of the DCELANM-Net framework in a variety of diverse environments. The results are shown in Table 4 .

In our studies, we compared the performances of ELAN, DCELAN, and DCELAN + Micro-MAE on two different datasets.

These datasets were CVC-ClinicDB and Kvasir-SEG #OTHEREFR , respectively.

When compared to ELAN, the performance of mDice and mIOU under DCELAN+Micro-MAE was enhanced by 6.77% and 5.37%, respectively.

DCELAN performed significantly better than ELAN, mDice, and mIOU on the Kvasir-SEG #OTHEREFR dataset, with a 6.03% and 5.43% improvement, respectively.

When compared to ELAN, mDice, and mIOU, the performance of DCELAN+Micro-MAE was enhanced by 7.90% and 5.78%, respectively.

As a result, it is possible to demonstrate that the performance of the combined DCELANM-Net framework is better when compared.

Dcelanm-Net:Medical Image Segmentation Based On Dual Channel Efficient Layer Aggregation Network With Learner

Abstract: Throughout the writing of this dissertation I have received a great deal of support and assistance. I would first like to thank my supervisor, Jun Su, whose expertise was invaluable in formulating the research questions and methodology. Your insightful feedback pushed me to sharpen my thinking and brought my work to a higher level. I would particularly like to acknowledge my teammate/group mate/team members, Zhangrun Xia, Krzysztof Przystupa and Orest Kochan for their wonderful collaboration and patient support Abstract. Segmenting medical images is a principal component of computer vision. The UNet model framework has taken over as the standard framework for this activity across a wide range of medical picture segmentation applications. Due to CNN's convolution operation limitations, the model's global modeling ability isn't absolutely perfect. Moreover, a single convolution operation cannot gather feature information at various scales, which will have an impact on the quality of the global feature extraction as well as the localization of local details. The DCELANM-Net structure, which this article offers, is a model that ingeniously combines a Dual Channel Efficient Layer Aggregation Network (DCELAN) and a Micro Masked Autoencoder (Micro-MAE). On the one hand, for the DCELAN, the features are more effectively fitted by deepening the network structure; the deeper network can successfully learn and fuse the features, which can more accurately locate the local feature information; and the utilization of each layer of channels is more effectively improved by widening

An Energy And Gpu-Computation Efficient Backbone Network For Real-Time Object Detection

Abstract: As DenseNet conserves intermediate features with diverse receptive fields by aggregating them with dense connection, it shows good performance on the object detection task. Although feature reuse enables DenseNet to produce strong features with a small number of model parameters and FLOPs, the detector with DenseNet backbone shows rather slow speed and low energy efficiency. We find the linearly increasing input channel by dense connection leads to heavy memory access cost, which causes computation overhead and more energy consumption. To solve the inefficiency of DenseNet, we propose an energy and computation efficient architecture called VoVNet comprised of One-Shot Aggregation (OSA). The OSA not only adopts the strength of DenseNet that represents diversified features with multi receptive fields but also overcomes the inefficiency of dense connection by aggregating all features only once in the last feature maps. To validate the effectiveness of VoVNet as a backbone network, we design both lightweight and large-scale VoVNet and apply them to one-stage and two-stage object detectors. Our VoVNet based detectors outperform DenseNet based ones with 2x faster speed and the energy consumptions are reduced by 1.6x - 4.1x. In addition to DenseNet, VoVNet also outperforms widely used ResNet backbone with faster speed and better energy efficiency. In particular, the small object detection performance has been significantly improved over DenseNet and ResNet.


In the context of networked feedback systems, the AoI-based policies have been proposed for solving resource allocation and enduser uncertainty reduction problems as in #REFR .

Further, we improve upon the convergence guarantees in #OTHEREFR for the case where the network is erasure free by proposing a novel scheduling policy.

Finally, we employ this policy to construct an approximate Nash solution for the finite-agent consensus problem.

In literature, the early works #OTHEREFR , #OTHEREFR have dealt with an optimal control problem with unreliable communication, albeit, for a single agent system and an unconstrained network under the TCP and the UDP communication protocols.

The work #OTHEREFR extends the setting to multi-agent games; however, the considered network is unconstrained.

In order to measure timeliness in communication networks, age of information (AoI) has been introduced as a potential metric.

Recently, age of incorrect information (AoII) is proposed for solving multiagent remote state estimation problems #OTHEREFR .

Age-optimal scheduling policies have been considered with Markovian error-prone channel state in #OTHEREFR , #OTHEREFR , with unknown erasure probabilities in #OTHEREFR , and over erroneous broadcast channels in #OTHEREFR .

A more detailed literature review on age-optimal scheduling policies can be found in #OTHEREFR .

To appropriately handle the concerns of increasing network interactions, one of the most relevant framework is that of mean-field games (MFGs) #OTHEREFR , #OTHEREFR .

It leads one to circumvent the issues posed by scalability, by allowing for a representative agent to play against the population, although, at the cost of entailing an approximate equilibrium solution to the finite-agent consensus problem.

Large Population Games On Constrained Unreliable Networks

Abstract: This paper studies an N-agent cost-coupled game where the agents are connected via an unreliable capacity constrained network. Each agent receives state information over that network which loses packets with probability p. A Base station (BS) actively schedules agent communications over the network by minimizing a weighted Age of Information (WAoI) based cost function under a capacity limit C < N on the number of transmission attempts at each instant. Under a standard information structure, we show that the problem can be decoupled into a scheduling problem for the BS and a game problem for the N agents. Since the scheduling problem is an NP hard combinatorics problem, we propose an approximately optimal solution which approaches the optimal solution as N → ∞. In the process, we also provide some insights on the case without channel erasure. Next, to solve the large population game problem, we use the mean-field game framework to compute an approximate decentralized Nash equilibrium. Finally, we validate the theoretical results using a numerical example.

Optimal Scheduling For Discounted Age Penalty Minimization In Multi-Loop Networked Control

Abstract: Age-of-information (AoI) is a metric quantifying information freshness at the receiver. Since AoI combines packet generation frequency, packet loss, and delay into a single metric, it has received a lot of research attention as an interface between communication network and application. In this work, we apply AoI to the problem of wireless scheduling for multi-loop networked control systems (NCS), i.e., feedback control loops closed over a shared wireless network. We model the scheduling problem as a Markov decision process (MDP) with AoI as its observable states and derive a relation of control system error and AoI. We further derive a stationary scheduling policy to minimize control error over an infinite horizon. We show that our scheduler outperforms the state-of-the-art scheduling policies for NCS. To the best of our knowledge, this is the first work proposing an AoI-based wireless scheduling policy that minimizes the control error over an infinite horizon for multi-loop NCS.


Typically, the prototypes µ 1 w and µ 2 w are determined through aggregation by averaging over Φ 1 w and Φ 2 w , respectively (e.g., #REFR ).

The shift s w is measured as the cosine distance (CD) between the word prototypes µ 1 w , µ 2 w as follows:

where CS is the cosine similarity between the prototypes.

Intuitively, the greater the CD(µ 1 w , µ 2 w ), the greater the shift in the dominant sense of w.

As a difference, in #OTHEREFR , the prototype embedding µ 2 w at time step t = 2 is computed by updating the prototype embedding µ 1 w at time step t = 1 through a weighted running average (e.g., #OTHEREFR ).

[2020b] , the CD metric is employed in a multilingual experiment where the shift is measured across a diachronic corpus with texts of different languages. This is the only example of cross-language shift detection.

CD is also used in time-aware approaches.

The integration of extra-linguistic information into word embeddings, such as time and social space, has been proposed in previous work based on static models #OTHEREFR Blei, 2018, Zeng et al., 2018] .

A Survey On Contextualised Semantic Shift Detection

Abstract: Semantic Shift Detection (SSD) is the task of identifying, interpreting, and assessing the possible change over time in the meanings of a target word. Traditionally, SSD has been addressed by linguists and social scientists through manual and time-consuming activities. In the recent years, computational approaches based on Natural Language Processing and word embeddings gained increasing attention to automate SSD as much as possible. In particular, over the past three years, significant advancements have been made almost exclusively based on word contextualised embedding models, which can handle the multiple usages/meanings of the words and better capture the related semantic shifts. In this paper, we survey the approaches based on contextualised embeddings for SSD (i.e., CSSDetection) and we propose a classification framework characterised by meaning representation, time-awareness, and learning modality dimensions. The framework is exploited i) to review the measures for shift assessment, ii) to compare the approaches on performance, and iii) to discuss the current issues in terms of scalability, interpretability, and robustness. Open challenges and future research directions about CSSDetection are finally outlined.

Leveraging Contextual Embeddings For Detecting Diachronic Semantic Shift

Abstract: We propose a new method that leverages contextual embeddings for the task of diachronic semantic shift detection by generating time specific word representations from BERT embeddings. The results of the experiments in the domain specific LiverpoolFC corpus suggest that the proposed method has performance comparable to the current state-of-the-art without requiring any time consuming domain adaptation on a large corpora. The results on a newly created Brexit news corpus suggest that the method can be successfully used for the detection of a short-term yearly semantic shift. And lastly, the model also shows promising results in a multilingual settings, where the task was to detect differences and similarities between diachronic semantic shifts in different languages.