We build our method upon DAPG #REFR , which is a two-stage learning strategy of behavior cloning and DRL for a grasping policy.

Unigrasp #OTHEREFR introduced the URDF model of the hand so that the work can be applied to different multifingered dexterous hands without retraining the model.

The hand-object relationship mainly uses more global information, such as the distance between hand and object #OTHEREFR , #OTHEREFR , motor signals #OTHEREFR , etc.

Though these descriptions work well in low-DoF grippers, they are limited for high-DoF dexterous grasping skills learning as restricted in capturing the interactions between the diverse object surfaces and complex hand articulations.

ManipNet #OTHEREFR introduces handobject spatial representations to predict object manipulation trajectories given wrist trajectories by supervised learning for animations.

Inspired by it, we propose to design more comprehensive hand-object interaction representations to empower reinforcement learning for dexterous grasping skills. Fig.2 shows the pipeline of our method.

The state-action pairs for our policy learning are prepared by retargeting the human grasp trajectories to a dexterous robotic hand.

Therefore, our method consists of three stages: retargeting, behavior cloning, and reinforcement learning, which is elaborated in Section III-B.

Our proposed state representation of the hand-object interaction for the policy is detailed in Section III-A.

Dexrepnet: Learning Dexterous Robotic Grasping Network With Geometric And Spatial Hand-Object Representations

Abstract: Robotic dexterous grasping is a challenging problem due to the high degree of freedom (DoF) and complex contacts of multi-fingered robotic hands. Existing deep reinforcement learning (DRL) based methods leverage human demonstrations to reduce sample complexity due to the high dimensional action space with dexterous grasping. However, less attention has been paid to hand-object interaction representations for high-level generalization. In this paper, we propose a novel geometric and spatial hand-object interaction representation, named DexRep, to capture dynamic object shape features and the spatial relations between hands and objects during grasping. DexRep comprises Occupancy Feature for rough shapes within sensing range by moving hands, Surface Feature for changing hand-object surface distances, and Local-Geo Feature for local geometric surface features most related to potential contacts. Based on the new representation, we propose a dexterous deep reinforcement learning method to learn a generalizable grasping policy DexRepNet. Experimental results show that our method outperforms baselines using existing representations for robotic grasping dramatically both in grasp success rate and convergence speed. It achieves a 93% grasping success rate on seen objects and higher than 80% grasping success rates on diverse objects of unseen categories in both simulation and real-world experiments.

Learning Complex Dexterous Manipulation With Deep Reinforcement Learning And Demonstrations

Abstract: Dexterous multi-fingered hands are extremely versatile and provide a generic way to perform multiple tasks in human-centric environments. However, effectively controlling them remains challenging due to their high dimensionality and large number of potential contacts. Deep reinforcement learning (DRL) provides a model-agnostic approach to control complex dynamical systems, but has not been shown to scale to high-dimensional dexterous manipulation. Furthermore, deployment of DRL on physical systems remains challenging due to sample inefficiency. Thus, the success of DRL in robotics has thus far been limited to simpler manipulators and tasks. In this work, we show that model-free DRL with natural policy gradients can effectively scale up to complex manipulation tasks with a high-dimensional 24-DoF hand, and solve them from scratch in simulated experiments. Furthermore, with the use of a small number of human demonstrations, the sample complexity can be significantly reduced, and enable learning within the equivalent of a few hours of robot experience. We demonstrate successful policies for multiple complex tasks: object relocation, in-hand manipulation, tool use, and door opening.


Our model is subjected to augmentation procedures to enhance its generalization, in line with the approach used in #REFR .

The Adam optimization algorithm is used to minimize the loss function with β 1 = 0.9, β 2 = 0.999, and = 10 −10 as the hyperparameters.

The initial values of the loss parameters are established based on the characteristics of the dataset utilized.

The batch size is set to 8, and the initial learning rate is λ = 10 −4 .

The learning rate decreases by 25% every 100 epochs for indoor localization and every 200 epochs for outdoor localization, with a maximum of 300 and 1000 epochs, respectively.

Additionally, a weight decay of 10 −4 and a dropout of p = 0.1 are applied to the encoders during training.

During training, the image is resized so that its smaller edge is resized to 256 pixels and a random 224 × 224 crop is taken.

Random adjustments are made to the brightness, contrast, and saturation of the image.

During testing, the image is rescaled with the smaller edge resized to 256 pixels, and a center crop is taken without any further augmentation.

The models were trained on multiple NVIDIA 2080T i GPUs and the PyTorch framework.

Hyperpose: Camera Pose Localization Using Attention Hypernetworks

Abstract: In this study, we propose the use of attention hypernetworks in camera pose localization. The dynamic nature of natural scenes, including changes in environment, perspective, and lighting, creates an inherent domain gap between the training and test sets that limits the accuracy of contemporary localization networks. To overcome this issue, we suggest a camera pose regressor that integrates a hypernetwork. During inference, the hypernetwork generates adaptive weights for the localization regression heads based on the input image, effectively reducing the domain gap. We also suggest the use of a Transformer-Encoder as the hypernetwork, instead of the common multilayer perceptron, to derive an attention hypernetwork. The proposed approach achieves superior results compared to state-of-the-art methods on contemporary datasets. To the best of our knowledge, this is the first instance of using hypernetworks in camera pose regression, as well as using Transformer-Encoders as hypernetworks. We make our code publicly available 1 .

Posenet: A Convolutional Network For Real-Time 6-Dof Camera Relocalization

Abstract: We present a robust and real-time monocular six degree of freedom relocalization system. Our system trains a convolutional neural network to regress the 6-DOF camera pose from a single RGB image in an end-to-end manner with no need of additional engineering or graph optimisation. The algorithm can operate indoors and outdoors in real time, taking 5ms per frame to compute. It obtains approximately 2m and 3 degrees accuracy for large scale outdoor scenes and 0.5m and 5 degrees accuracy indoors. This is achieved using an efficient 23 layer deep convnet, demonstrating that convnets can be used to solve complicated out of image plane regression problems. This was made possible by leveraging transfer learning from large scale classification data. We show that the PoseNet localizes from high level features and is robust to difficult lighting, motion blur and different camera intrinsics where point based SIFT registration fails. Furthermore we show how the pose feature that is produced generalizes to other scenes allowing us to regress pose with only a few dozen training examples.


One crucial tool for utilizing this knowledge is the affinity matrix #REFR , where each element indicates the degree of association between two key samples.

The presence of redundant information is attributed to the random sampling method used in key sample acquisition, which results in the inclusion of non-key samples.

Additionally, there may be mutual exclusivity between different key sample evaluation metrics, as the determination of whether a sample is a key sample or not may yield opposite results under different metrics.

Our proposed solution is to model the correlation between the bag and the key instance set as well as the key bag set by using affinity matrixes.

We then employ a mask to eliminate certain features and obtain the fusion bag.

Specifically, we have now obtained three abstract knowledge B * i , I * , and B * , which contain valuable information, such as instances that trigger bag labels #OTHEREFR and bags that can build embedding functions #OTHEREFR .

Specifically, the affinity matrix A I i between a given bag B i = {x ij } ni j=1 and I * is defined as:

where x * ij ∈ B * and x * k ∈ I * are the deep-level representations of x ij ∈ B i and x ins k ∈ I, respectively. Similarly, the affinity matrix A B

Data-Driven Knowledge Fusion For Deep Multi-Instance Learning

Abstract: Multi-instance learning (MIL) is a widely-applied technique in practical applications that involve complex data structures. MIL can be broadly categorized into two types: traditional methods and those based on deep learning. These approaches have yielded significant results, especially with regards to their problem-solving strategies and experimental validation, providing valuable insights for researchers in the MIL field. However, a considerable amount of knowledge is often trapped within the algorithm, leading to subsequent MIL algorithms that solely rely on the model's data fitting to predict unlabeled samples. This results in a significant loss of knowledge and impedes the development of more intelligent models. In this paper, we propose a novel data-driven knowledge fusion for deep multi-instance learning (DKMIL) algorithm. DKMIL adopts a completely different idea from existing deep MIL methods by analyzing the decision-making of key samples in the data set (referred to as the data-driven) and using the knowledge fusion module designed to extract valuable information from these samples to assist the model's training. In other words, this module serves as a new interface between data and the model, providing strong scalability and enabling the use of prior knowledge from existing algorithms to enhance the learning ability of the model. Furthermore, to adapt the downstream modules of the model to more knowledge-enriched features extracted from the datadriven knowledge fusion module, we propose a two-level attention module that gradually learns shallow-and deep-level features of the samples to achieve more effective classification. We will prove the scalability of the knowledge fusion module while also verifying the efficacy of the proposed architecture by conducting experiments on 38 data sets across 6 categories.

Multi-Instance Learning By Treating Instances As Non-I.I.D. Samples

Abstract: Previous studies on multi-instance learning typically treated instances in the bags as independently and identically distributed. The instances in a bag, however, are rarely independent in real tasks, and a better performance can be expected if the instances are treated in an non-i.i.d. way that exploits relations among instances. In this paper, we propose two simple yet effective methods. In the first method, we explicitly map every bag to an undirected graph and design a graph kernel for distinguishing the positive and negative bags. In the second method, we implicitly construct graphs by deriving affinity matrices and propose an efficient graph kernel considering the clique information. The effectiveness of the proposed methods are validated by experiments.


We further note from Table 8 that for Gaussian smoothing #REFR , reducing the clean accuracy below 60% by increasing the noise level results in a drop in ACR, which does not happen in the proposed method.

As discussed in Section 4.5, we compare across different methods against fixed levels of clean accuracy set to the following -45%, 50%, 55%, 60%, 65%, 70% and 75%.

As shown in Table 8 , the proposed approach has significantly higher ACR in all cases when compared to the baselines.

The training parameters σ N and σ U are chosen to set the kurtosis of the Normal-Uniform distribution to a negative value (−0.22), which gives the relation between σ N and σ U as σ U = √ 3σ N /2.

We further tune the hyper-parameter (σ N ) to achieve the required level of clean accuracy.

Experimentally, a low value of β ∈ {2, 3, 4} for our proposed regularization (Equation 3) gives the best performance for the considered levels of clean accuracy.

In supplementary material, we present a detailed ablation study of the proposed method.

We present the impact of variation in the hyperparameter β, impact of choice of regularizer and the effect of Kurtosis on the final performance of the certified classifier in terms of its clean accuracy, ℓ 1 ACR and ℓ 2 ACR.

Certified Adversarial Robustness Within Multiple Perturbation Bounds

Abstract: Randomized smoothing (RS) is a well known certified defense against adversarial attacks, which creates a smoothed classifier by predicting the most likely class under random noise perturbations of inputs during inference. While initial work focused on robustness to ℓ 2 norm perturbations using noise sampled from a Gaussian distribution, subsequent works have shown that different noise distributions can result in robustness to other ℓ p norm bounds as well. In general, a specific noise distribution is optimal for defending against a given ℓ p norm based attack. In this work, we aim to improve the certified adversarial robustness against multiple perturbation bounds simultaneously. Towards this, we firstly present a novel certification scheme, that effectively combines the certificates obtained using different noise distributions to obtain optimal results against multiple perturbation bounds. We further propose a novel training noise distribution along with a regularized training scheme to improve the certification within both ℓ 1 and ℓ 2 perturbation norms simultaneously. Contrary to prior works, we compare the certified robustness of different training algorithms across the same natural (clean) accuracy, rather than across fixed noise levels used for training and certification. We also empirically invalidate the argument that training and certifying the classifier with the same amount of noise gives the best results. The proposed approach achieves improvements on the ACR (Average Certified Radius) metric across both ℓ 1 and ℓ 2 perturbation bounds. Code available at https://github.com/ val-iisc/NU-Certified-Robustness

Certified Adversarial Robustness Via Randomized Smoothing

Abstract: We show how to turn any classifier that classifies well under Gaussian noise into a new classifier that is certifiably robust to adversarial perturbations under the $\ell_2$ norm. This "randomized smoothing" technique has been proposed recently in the literature, but existing guarantees are loose. We prove a tight robustness guarantee in $\ell_2$ norm for smoothing with Gaussian noise. We use randomized smoothing to obtain an ImageNet classifier with e.g. a certified top-1 accuracy of 49% under adversarial perturbations with $\ell_2$ norm less than 0.5 (=127/255). No certified defense has been shown feasible on ImageNet except for smoothing. On smaller-scale datasets where competing approaches to certified $\ell_2$ robustness are viable, smoothing delivers higher certified accuracies. Our strong empirical results suggest that randomized smoothing is a promising direction for future research into adversarially robust classification. Code and models are available at this http URL.


For each content feature, we encoded tweets with Sentence-BERT (SBERT) #REFR and compute their average embeddings as a user-level representation.

We use users' stances on wearing masks and racial equality as features to build a prediction model for their stance on Trump.

In addition, we extract 1) content features from their historical tweets and 2) content-agnostic features from their profile and Twitterrelated statistics to perform an ablation study.

7.1.1 Content Features. We used three content features for Trump stance prediction.

We extracted mask and racial equality-related content features using their respective topic-related tweets.

We also extracted historical content features using each user's most recent 100 tweets, excluding all topic-related tweets.

Twitter profile features were extracted from the profile description and URL listed in each user's profile.

They were both encoded using SBERT and averaged to obtain user-level content-agnostic representation.

Twitter statistics features include: 1) number of followers, 2) number of followings, 3) number of tweets posted by the user, 4) number of public lists that the user is a member of, 5) number of days since the creation of account, 6) whether the account protects its tweets, and 7) whether the account is verified.

Wearing Masks Implies Refuting Trump?: Towards Target-Specific User Stance Prediction Across Events In Covid-19 And Us Election 2020

Abstract: People who share similar opinions towards controversial topics could form an echo chamber and may share similar political views toward other topics as well. The existence of such connections, which we call connected behavior, gives researchers a unique opportunity to predict how one would behave for a future event given their past behaviors. In this work, we propose a framework to conduct connected behavior analysis. Neural stance detection models are trained on Twitter data collected on three seemingly independent topics, i.e., wearing a mask, racial equality, and Trump, to detect people's stance, which we consider as their online behavior in each topic-related event. Our results reveal a strong connection between the stances toward the three topical events and demonstrate the power of past behaviors in predicting one's future behavior. CCS CONCEPTS • Applied computing → Sociology; • Computing methodologies → Natural language processing.

Sentence-Bert: Sentence Embeddings Using Siamese Bert-Networks

Abstract: BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019) has set a new state-of-the-art performance on sentence-pair regression tasks like semantic textual similarity (STS). However, it requires that both sentences are fed into the network, which causes a massive computational overhead: Finding the most similar pair in a collection of 10,000 sentences requires about 50 million inference computations (~65 hours) with BERT. The construction of BERT makes it unsuitable for semantic similarity search as well as for unsupervised tasks like clustering. ::: In this publication, we present Sentence-BERT (SBERT), a modification of the pretrained BERT network that use siamese and triplet network structures to derive semantically meaningful sentence embeddings that can be compared using cosine-similarity. This reduces the effort for finding the most similar pair from 65 hours with BERT / RoBERTa to about 5 seconds with SBERT, while maintaining the accuracy from BERT. ::: We evaluate SBERT and SRoBERTa on common STS tasks and transfer learning tasks, where it outperforms other state-of-the-art sentence embeddings methods.


Similarly to the observations of Schöller et al. #REFR , we see that S-GAN's performance is mixed.

#OTHEREFR compared the Average Displacement Error (ADE) and the Final Displacement Error (FDE) of S-GANbased prediction against CV prediction and CV prediction with added noise (CVN), showing that the latter ones perform comparably across the scenes in the ETH #OTHEREFR and UCY #OTHEREFR datasets. In Fig.

2 , we compare their multistep prediction performance (i.e., the L2-norm between the predicted position and the ground truth at each timestep of prediction), which is informative for navigation tasks.

While it exhibits lower error on Zara1, it ties with CVN and CV on ETH-Uni and Zara2 and it is outperformed by them in Hotel, whereas on UCY-Uni all models perform comparably.

It should be noted that the human behavior featured in these datasets mostly consists of linear segments that can be well approximated by CV/CVN whereas the S-GAN models promise a better generalization to more complex, nonlinear behavior.

From Crowd Motion Prediction To Robot Navigation In Crowds

Abstract: We focus on robot navigation in crowded environments. To navigate safely and efficiently within crowds, robots need models for crowd motion prediction. Building such models is hard due to the high dimensionality of multiagent domains and the challenge of collecting or simulating interaction-rich crowd-robot demonstrations. While there has been important progress on models for offline pedestrian motion forecasting, transferring their performance on real robots is nontrivial due to close interaction settings and novelty effects on users. In this paper, we investigate the utility of a recent state-of-the-art motion prediction model (S-GAN) for crowd navigation tasks. We incorporate this model into a model predictive controller (MPC) and deploy it on a self-balancing robot which we subject to a diverse range of crowd behaviors in the lab. We demonstrate that while S-GAN motion prediction accuracy transfers to the real world, its value is not reflected on navigation performance, measured with respect to safety and efficiency; in fact, the MPC performs indistinguishably even when using a simple constantvelocity prediction model, suggesting that substantial model improvements might be needed to yield significant gains for crowd navigation tasks. Footage from our experiments can be found at https://youtu.be/mzFiXg8KsZ0.

What The Constant Velocity Model Can Teach Us About Pedestrian Motion Prediction

Abstract: Pedestrian motion prediction is a fundamental task for autonomous robots and vehicles to operate safely. In recent years many complex approaches based on neural networks have been proposed to address this problem. In this work we show that – surprisingly – a simple Constant Velocity Model can outperform even state-of-the-art neural models. This indicates that either neural networks are not able to make use of the additional information they are provided with, or that this information is not as relevant as commonly believed. Therefore, we analyze how neural networks process their input and how it impacts their predictions. Our analysis reveals pitfalls in training neural networks for pedestrian motion prediction and clarifies false assumptions about the problem itself. In particular, neural networks implicitly learn environmental priors that negatively impact their generalization capability, the motion history of pedestrians is irrelevant and interactions are too complex to predict. Our work shows how neural networks for pedestrian motion prediction can be thoroughly evaluated and our results indicate which research directions for neural motion prediction are promising in future.


For autoregressive S4, when we followed the Long-T5 method, we found it to highly unstable and failed to converge, similar to what was found in the literature #REFR on its dependency certain keywords e.g., true/false.

Autoregressive re-ranker. We also consider autoregressive re-rankers #OTHEREFR .

For a transformer baseline, we use a pre-trained LongT5 #OTHEREFR .

The query and the document are concatenated together to form the input sequence: Query: d Document: q i Relevant:, which is fed into the encoder.

The decoder then predicts true for relevant documents or false for irrelevant documents.

During inference, a softmax function is applied to the logits of the true and the false tokens to normalize the results across multiple queries.

Therefore, we followed the same setting as in the cross-encoder, except that the underlying model is autoregressive rather than bi-directional.

Here, the concatenated dialogue and a candidate question are fed into the S4 re-ranker and the average pooling of the last layer is classified as either relevant or irrelevant through a classification head.

Dialogue-Contextualized Re-Ranking For Medical History-Taking

Abstract: AI-driven medical history-taking is an important component in symptom checking, automated patient intake, triage, and other AI virtual care applications. As historytaking is extremely varied, machine learning models require a significant amount of data to train. To overcome this challenge, existing systems are developed using indirect data or expert knowledge. This leads to a training-inference gap as models are trained on different kinds of data than what they observe at inference time. In this work, we present a two-stage re-ranking approach that helps close the training-inference gap by re-ranking the first-stage question candidates using a dialogue-contextualized model. For this, we propose a new model, global re-ranker, which cross-encodes the dialogue with all questions simultaneously, and compare it with several existing neural baselines. We test both transformer and S4-based language model backbones. We find that relative to the expert system, the best performance is achieved by our proposed global re-ranker with a transformer backbone, resulting in a 30% higher normalized discount cumulative gain (nDCG) and a 77% higher mean average precision (mAP). As part of this work, we also release pre-trained checkpoints for bi-directional and autoregressive S4 models trained on Wikipedia and PubMed data.

Document Ranking With A Pretrained Sequence-To-Sequence Model

Abstract: This work proposes a novel adaptation of a pretrained sequence-to-sequence model to the task of document ranking. Our approach is fundamentally different from a commonly-adopted classification-based formulation of ranking, based on encoder-only pretrained transformer architectures such as BERT. We show how a sequence-to-sequence model can be trained to generate relevance labels as"target words", and how the underlying logits of these target words can be interpreted as relevance probabilities for ranking. On the popular MS MARCO passage ranking task, experimental results show that our approach is at least on par with previous classification-based models and can surpass them with larger, more-recent models. On the test collection from the TREC 2004 Robust Track, we demonstrate a zero-shot transfer-based approach that outperforms previous state-of-the-art models requiring in-dataset cross-validation. Furthermore, we find that our approach significantly outperforms an encoder-only model in a data-poor regime (i.e., with few training examples). We investigate this observation further by varying target words to probe the model's use of latent knowledge.


Connection To Skip-Gram Embedding Methods

As in Table 1 in #REFR , we can expect the same theoretical properties w.r.t.

This theorem shows that the node with the highest bridgeness inG r is also the node with the largest importance score (thus, node-level explanation for W) for the DeepWalk embedding over G.

In the Appendix, we outline how the same node is also likely to have the highest bridgeness score in G (i.e., arg max

v j ∈V Bridgeness(v j |C,G r ) arg max v j ∈V Bridgeness(v j |C, G)).

Other embedding models such as LINE, PTE, and struc2vec can be analyzed in the same way.

For LINE and PTE, similar theoretical approximations were studied in #OTHEREFR .

perturbation for both LINE #OTHEREFR and PTE #OTHEREFR because our perturbation method does not change degree matrices and other constants.

struc2vec #OTHEREFR learns low-dimensional representations using a Skip-gram architecture on the context graph, which is constructed by structural similarity among nodes.

When input adjacency is assumed as the context graph, its bridge nodes are still important as in DeepWalk.

Similarly, RWJBG #OTHEREFR learns node embeddings on joint behavior graphs under the same Skip-gram framework and our analysis remains applicable.

UserItem2vec #OTHEREFR also constructs an attributed heterogeneous user/item network and the embeddings are learned on the Skip-gram architecture; the most important nodes are identifiable under our proposed idea.

Generating Post-Hoc Explanations For Skip-Gram-Based Node Embeddings By Identifying Important Nodes With Bridgeness

Abstract: Node representation learning in a network is an important machine learning technique for encoding relational information in a continuous vector space while preserving the inherent properties and structures of the network. Recently, unsupervised node embedding methods such as DeepWalk [1], LINE [2], struc2vec [3], PTE [4], UserItem2vec [5], and RWJBG [6] have emerged from the Skip-gram model [7] and perform better performance in several downstream tasks such as node classification and link prediction than the existing relational models. However, providing posthoc explanations of Skip-gram-based embeddings remains a challenging problem because of the lack of explanation methods and theoretical studies applicable for embeddings. In this paper, we first show that global explanations to the Skip-gram-based embeddings can be found by computing bridgeness under a spectral cluster-aware local perturbation. Moreover, a novel gradient-based explanation method, which we call GRAPH-wGD, is proposed that allows the top-q global explanations about learned graph embedding vectors more efficiently. Experiments show that the ranking of nodes by scores using GRAPH-wGD is highly correlated with true bridgeness scores. We also observe that the top-q node-level explanations selected by GRAPH-wGD have higher importance scores and produce more changes in class label prediction when perturbed, compared with the nodes selected by recent alternatives, using five real-world graphs.

Network Embedding As Matrix Factorization: Unifying Deepwalk, Line, Pte, And Node2Vec

Abstract: Since the invention of word2vec, the skip-gram model has significantly advanced the research of network embedding, such as the recent emergence of the DeepWalk, LINE, PTE, and node2vec approaches. In this work, we show that all of the aforementioned models with negative sampling can be unified into the matrix factorization framework with closed forms. Our analysis and proofs reveal that: (1) DeepWalk empirically produces a low-rank transformation of a network's normalized Laplacian matrix; (2) LINE, in theory, is a special case of DeepWalk when the size of vertices' context is set to one; (3) As an extension of LINE, PTE can be viewed as the joint factorization of multiple networks» Laplacians; (4) node2vec is factorizing a matrix related to the stationary distribution and transition probability tensor of a 2nd-order random walk. We further provide the theoretical connections between skip-gram based network embedding algorithms and the theory of graph Laplacian. Finally, we present the NetMF method as well as its approximation algorithm for computing network embedding. Our method offers significant improvements over DeepWalk and LINE for conventional network mining tasks. This work lays the theoretical foundation for skip-gram based network embedding methods, leading to a better understanding of latent network representation learning.


In [25, Theorem 2.1], Moeller and Ullrich proved the following concentration inequality for the smallest and largest eigenvalue of such random Gram matrices. The result is based on Tropp #REFR .

(Lx) j = d k=1 f | e k e k (ξ j ) = f (ξ j )

for j = 1, . . . , N . Thus the inequality (3.1) can be rewritten as

Obviously, the best possible constants 1 ± η in (3.3) are given by the minimal resp. maximal eigenvalue of 1 N L * L.

set, a probability measure on Ω and (e k ) D k=1 be an orthonormal system in L 2 (Ω, ). Let ξ 1 , . . . , ξ N ∈ Ω be drawn i.i.d.

according to , L = (e k (ξ j )) N,D j,k=1 , and P = ⊗N the product measure.

Then the following concentration inequalities for the extremal eigenvalues of 1

Marcinkiewicz-Zygmund Inequalities For Scattered And Random Data On The Q-Sphere

Abstract: The recovery of multivariate functions and estimating their integrals from finitely many samples is one of the central tasks in modern approximation theory. Marcinkiewicz-Zygmund inequalities provide answers to both the recovery and the quadrature aspect. In this paper, we put ourselves on the q-dimensional sphere S q , and investigate how well continuous Lp-norms of polynomials f of maximum degree n on the sphere S q can be discretized by positively weighted Lp-sum of finitely many samples, and discuss the relationship between the offset between the continuous and discrete quantities, the number and distribution of the (deterministic or randomly chosen) sample points ξ 1 ,. .. , ξ N on S q , the dimension q, and the polynomial degree n.

User-Friendly Tail Bounds For Sums Of Random Matrices

Abstract: This paper presents new probability inequalities for sums of independent, random, self-adjoint matrices. These results place simple and easily verifiable hypotheses on the summands, and they deliver strong conclusions about the large-deviation behavior of the maximum eigenvalue of the sum. Tail bounds for the norm of a sum of random rectangular matrices follow as an immediate corollary. The proof techniques also yield some information about matrix-valued martingales. ::: ::: In other words, this paper provides noncommutative generalizations of the classical bounds associated with the names Azuma, Bennett, Bernstein, Chernoff, Hoeffding, and McDiarmid. The matrix inequalities promise the same diversity of application, ease of use, and strength of conclusion that have made the scalar inequalities so valuable.

following concentration inequality
random Gram matrices


This task is designed to be extremely difficult for the vanilla RNN due to the VGP #REFR .

Copy memory is a synthetic many-to-many classification task first introduced in #OTHEREFR to test the ability to recall information bits after a very long delay.

Input samples contain the alphabets 2-9, and the blank and start letters 0, 1, are one-hot encoded into vectors of dim 10.

The recalling sequence A contains the first K letters of the input and are uniformly sampled from the alphabet.

It is followed by L blank letters, 1 start letter, and another K − 1 blank letters.

An output sample contains K + L + 1 blanks letter and the recalling sequence A.

The baseline model output K + L + 1 blanks letter and another K − 1 letters sampled uniformly of the alphabet, which has the loss of K ln 8 L+2K .

Adaptive-Saturated Rnn: Remember More With Less Instability

Abstract: Orthogonal parameterization is a compelling solution to the vanishing gradient problem (VGP) in recurrent neural networks (RNNs). With orthogonal parameters and non-saturated activation functions, gradients in such models are constrained to unit norms. On the other hand, although the traditional vanilla RNNs are seen to have higher memory capacity, they suffer from the VGP and perform badly in many applications. This work proposes Adaptive-Saturated RNNs (asRNN), a variant that dynamically adjusts its saturation level between the two mentioned approaches. Consequently, asRNN enjoys both the capacity of a vanilla RNN and the training stability of orthogonal RNNs. Our experiments show encouraging results of asRNN on challenging sequence learning benchmarks compared to several strong competitors. The research code is accessible at https://github.com/ndminhkhoi46/asRNN/.

Recurrent Orthogonal Networks And Long-Memory Tasks

Abstract: Although RNNs have been shown to be powerful tools for processing sequential data, finding architectures or optimization strategies that allow them to model very long term dependencies is still an active area of research. In this work, we carefully analyze two synthetic datasets originally outlined in (Hochreiter and Schmidhuber, 1997) which are used to evaluate the ability of RNNs to store information over many time steps. We explicitly construct RNN solutions to these problems, and using these constructions, illuminate both the problems themselves and the way in which RNNs store different types of information in their hidden states. These constructions furthermore explain the success of recent methods that specify unitary initializations or constraints on the transition matrices.


An attention mechanism #REFR is equipped between the encoder and decoder, and features of each part of the image are selectively sent to the decoder.

We focused on three image captioning models.

None of them were fine-tuned using the data for the target task (because no image description text was available for the target task), and the original models were used as it is.

InceptionV3+RNN: This is a basic model of small scale that encodes an input image into a vector by using Incep-tionV3 #OTHEREFR and then decodes it to generate a caption using a recurrent neural network (Gated Recurrent Unit; GRU #OTHEREFR ).

The entire system was trained with the MS-COCO dataset #OTHEREFR . We followed Ten-sorFlow's tutorial implementation #OTHEREFR .

BLIP: This is a foundation model that has learned a large number of images and amount of text and is applicable to a wide range of Vision and Language tasks #OTHEREFR .

When used as an image captioner, it takes the form of an encoder-decoder configuration based on the Transformer model.

Users can easily run the sample code (demo.ipynb) on GitHub #OTHEREFR to obtain captions for their own images.

BLIP is a relatively advanced, large-scale model that is capable of producing quite accurate captions.

Image Captioners Sometimes Tell More Than Images They See

Abstract: Image captioning, a.k.a. "image-to-text," which generates descriptive text from given images, has been rapidly developing throughout the era of deep learning. To what extent is the information in the original image preserved in the descriptive text generated by an image captioner? To answer that question, we have performed experiments involving the classification of images from descriptive text alone, without referring to the images at all, and compared results with those from standard image-based classifiers. We have evaluate several image captioning models with respect to a disaster image classification task, CrisisNLP, and show that descriptive text classifiers can sometimes achieve higher accuracy than standard image-based classifiers. Further, we show that fusing an image-based classifier with a descriptive text classifier can provide improvement in accuracy.

Show, Attend And Tell: Neural Image Caption Generation With Visual Attention

Abstract: Inspired by recent work in machine translation and object detection, we introduce an attention based model that automatically learns to describe the content of images. We describe how we can train this model in a deterministic manner using standard backpropagation techniques and stochastically by maximizing a variational lower bound. We also show through visualization how the model is able to automatically learn to fix its gaze on salient objects while generating the corresponding words in the output sequence. We validate the use of attention with state-of-the-art performance on three benchmark datasets: Flickr9k, Flickr30k and MS COCO.


They suggested that, since the full-length sequence was used, the biologically important short subsequences that would be more powerful at representing the protein were lost due the low signal to noise ratio #REFR .

Furthermore, by using the obtained weights, the neural attention mechanism provides clear visualizations, which makes models easier to analyze even when modeling is performed using real-valued vector representations rather than discrete features.

In their experiments using three CPI datasets #OTHEREFR , they demonstrated that the proposed approach based on end-to-end learning of GNN and CNN can achieve competitive or higher performance than existing approaches.

WideDTA #OTHEREFR : They propose a methodology to predict protein-ligand binding affinity through text-only information of both proteins and compounds.

Without relying on 3D structure information of the complex or 2D

representation of the compound, they learn high dimensional features from sequences of the proteins and ligands.

In order to overcome this problem, they propose to integrate different pieces of text-based information in the WideDTA model to provide a better representation of the interaction.

They still utilize the protein sequence and ligand SMILES string by representing them as a set of words.

A word of a protein sequence corresponds to a three-residue subsequence,

whereas a word of a ligand is equal to an 8-character subsequence extracted with a sliding window approach #OTHEREFR .

In addition, they use two textual information sources that can provide valuable clues about the specificity of the interaction.

Resdta: Predicting Drug-Target Binding Affinity Using Residual Skip Connections

Abstract: I would like to thank Almighty as I have been able to successfully complete my thesis work. I am indebted to my supervisor, Professor Dr. Md. Aynal Haque, whose expertise was invaluable in formulating the research questions and methodology. I would like to thank him for his guidance, assistance and patience throughout this research. Without his support and encouragement, it would not have been possible to complete the research work. His insightful feedback pushed me to sharpen my thinking and brought this work to a higher level. I am very lucky that I have got the opportunity to work under his supervision. I would also like to thank my senior Udoy Kamal and friend Proteeti Ghosh to constantly support me throughout the thesis and motivated me a lot.

Deepdta: Deep Drug‐Target Binding Affinity Prediction

Abstract: Motivation The identification of novel drug‐target (DT) interactions is a substantial part of the drug discovery process. Most of the computational methods that have been proposed to predict DT interactions have focused on binary classification, where the goal is to determine whether a DT pair interacts or not. However, protein‐ligand interactions assume a continuum of binding strength values, also called binding affinity and predicting this value still remains a challenge. The increase in the affinity data available in DT knowledge‐bases allows the use of advanced learning techniques such as deep learning architectures in the prediction of binding affinities. In this study, we propose a deep‐learning based model that uses only sequence information of both targets and drugs to predict DT interaction binding affinities. The few studies that focus on DT binding affinity prediction use either 3D structures of protein‐ligand complexes or 2D features of compounds. One novel approach used in this work is the modeling of protein sequences and compound 1D representations with convolutional neural networks (CNNs). Results The results show that the proposed deep learning based model that uses the 1D representations of targets and drugs is an effective approach for drug target binding affinity prediction. The model in which high‐level representations of a drug and a target are constructed via CNNs achieved the best Concordance Index (CI) performance in one of our larger benchmark datasets, outperforming the KronRLS algorithm and SimBoost, a state‐of‐the‐art method for DT binding affinity prediction. Availability and implementation https://github.com/hkmztrk/DeepDTA Supplementary information Supplementary data are available at Bioinformatics online.


Later, an exhaustive search for finding the optimal rotation of the signal constellation was presented in #REFR .

These constellations are defined for point-to-point systems #OTHEREFR - #OTHEREFR and their constellation shaping is oblivious to interference.

Such an inability to respond to interference is an obstacle to improving the biterror rate and spectral efficiency of today's interference-limited communication systems.

In this paper, we consider the two-user one-sided IC, also known as the Z-interference channel (ZIC) #OTHEREFR , with imperfect CSI.

Previous works have examined the ZIC with finite alphabet inputs and uniform constellations in certain regimes.

In #OTHEREFR , it is shown that rotating one input constellation (alphabet) can improve the sum-rate of the two-user IC in strong/very strong interference regimes.

The focus of the above papers is to maximize the achievable rates, and they do not study bit-error rate (BER) performance.

BER is a critical metric, and interference can severely increase the BER by distorting the received constellation when uniform constellations like QAM are employed.

Deep autoencoder (DAE)-based end-to-end communication is an emerging approach to finite-alphabet communication in which BER is the main performance measure and constellation design is inherent to it.

Various groups have proposed DAEbased communication both for single-and multi-user systems #OTHEREFR - #OTHEREFR .

Particularly, #OTHEREFR , #OTHEREFR , #OTHEREFR have studied communication over the IC.

Interference-Aware Constellation Design For Z-Interference Channels With Imperfect Csi

Abstract: A deep autoencoder (DAE)-based end-to-end communication over the two-user Z-interference channel (ZIC) with finite-alphabet inputs is designed in this paper. The design is for imperfect channel state information (CSI) where both estimation and quantization errors exist. The proposed structure jointly optimizes the encoders and decoders to generate interferenceaware constellations that adapt their shape to the interference intensity in order to minimize the bit error rate. A normalization layer is designed to guarantee an average power constraint in the DAE while allowing the architecture to generate constellations with nonuniform shapes. This brings further shaping gain compared to standard uniform constellations such as quadrature amplitude modulation. The performance of the DAE-ZIC is compared with two conventional methods, i.e., standard and rotated constellations. The proposed structure significantly enhances the performance of the ZIC. Simulation results confirm bit error rate reduction in all interference regimes (weak, moderate, and strong). At a signal-to-noise ratio of 20dB, the improvements reach about two orders of magnitude when only quantization error exists, indicating that the DAE-ZIC is highly robust to the interference compared to the conventional methods.

Two-User Gaussian Interference Channel With Finite Constellation Input And Fdma

Abstract: In the two-user Gaussian Strong Interference Channel (GSIC) with finite constellation inputs, it is known that relative rotation between the constellations of the two users enlarges the Constellation Constrained (CC) capacity region. In this paper, a metric for finding the approximate angle of rotation (with negligibly small error) to maximally enlarge the CC capacity for the two-user GSIC is presented. In the case of Gaussian input alphabets with equal powers for both the users and the modulus of both the cross-channel gains being equal to unity, it is known that the FDMA rate curve touches the capacity curve of the GSIC. It is shown that, with unequal powers for both the users also, when the modulus of one of the cross-channel gains being equal to one and the modulus of the other cross-channel gain being greater than or equal to one, the FDMA rate curve touches the capacity curve of the GSIC. On the contrary, it is shown that, under finite constellation inputs, with both the users using the same constellation, the FDMA rate curve strictly lies within (never touches) the enlarged CC capacity region throughout the strong-interference regime. This means that using FDMA it is impossible to go close to the CC capacity. It is well known that for the Gaussian input alphabets, the FDMA inner-bound, at the optimum sum-rate point, is always better than the simultaneous-decoding inner-bound throughout the weak-interference regime. For a portion of the weak interference regime, it is shown that with identical finite constellation inputs for both the users, the simultaneous-decoding inner-bound, enlarged by relative rotation between the constellations, is strictly better than the FDMA inner-bound.


Our results were achieved with simple data augmentations, although using more extensive simulations, such as those in #REFR , could lead to greater generalizability.

For diffusion tensor fitting, Table 1 shows that the Swin model achieves lower MAE than BM4D in all metrics in white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) in both the HCP and TBI test datasets without finetuning.

For the SPIN dataset, the Swin model outperforms BM4D, except for FA estimation in GM and CSF; however, applying no denoising outperforms both BM4D and the Swin model for MD estimation in CSF possibly because of noise due to motion left uncorrected after Eddy. Qualitative comparison in Fig.

2 is consistent with these quantitative results and shows that the Swin model is able to capture more of the finer features in the WM and GM microstructure without the excessive smoothing of BM4D.

For 4 th and 6 th order spherical harmonic fitting, Table 2 shows that the Swin model achieves the lowest JSD in GM and CSF across all datasets.

The Swin model performs worse than MPPCA in WM for 6 th order spherical harmonic fitting, even in HCP data, possibly because the model is underfitted (due to compute constraints only five training epochs could be completed) and also because the Swin model denoises one direction at a time, whereas MPPCA is able to collectively denoise all directions.

In addition, by processing each dMRI volume separately, we are able to consider the full brain volume but do not utilize the correlation across volumes, such as the transformer patch-based approach in #OTHEREFR .

Due to GPU memory constraints, a trade-off exists between utilizing spatial and angular correlations.

Finally, Swin transformers can efficiently capture long-range dependencies, but require more extensive pretraining when compared to CNNs, which is integral to the success of our approach #OTHEREFR .

Generalized Diffusion Mri Denoising And Super-Resolution Using Swin Transformers

Abstract: Diffusion MRI is a non-invasive, in-vivo medical imaging method able to map tissue microstructure and structural connectivity of the human brain, as well as detect changes, such as brain development and injury, not visible by other clinical neuroimaging techniques. However, acquiring high signal-to-noise ratio (SNR) datasets with high angular and spatial sampling requires prohibitively long scan times, limiting usage in many important clinical settings, especially children, the elderly, and emergency patients with acute neurological disorders who might not be able to cooperate with the MRI scan without conscious sedation or general anesthesia. Here, we propose to use a Swin UNEt TRansformers (Swin UNETR) model, trained on augmented Human Connectome Project (HCP) data and conditioned on registered T1 scans, to perform generalized denoising and super-resolution of diffusion MRI invariant to acquisition parameters, patient populations, scanners, and sites. We qualitatively demonstrate super-resolution with artificially downsampled HCP data in normal adult volunteers. Our experiments on two other unrelated datasets, one of children with neurodevelopmental disorders and one of traumatic brain injury patients, show that our method demonstrates superior denoising despite wide data distribution shifts. Further improvement can be achieved via finetuning with just one additional subject. We apply our model to diffusion tensor (2 nd order spherical harmonic) and higher-order spherical harmonic coefficient estimation and show results superior to current state-of-the-art methods. Our method can be used out-of-the-box or minimally finetuned to denoise and superresolve a wide variety of diffusion MRI datasets. The code and model are publicly available at https://github.com/ucsfncl/dmri-swin.

Training A Neural Network For Gibbs And Noise Removal In Diffusion Mri

Abstract: We develop and evaluate a neural network-based method for Gibbs artifact and noise removal. A convolutional neural network (CNN) was designed for artifact removal in diffusion-weighted imaging data. Two implementations were considered: one for magnitude images and one for complex images. Both models were based on the same encoder-decoder structure and were trained by simulating MRI acquisitions on synthetic non-MRI images. Both machine learning methods were able to mitigate artifacts in diffusion-weighted images and diffusion parameter maps. The CNN for complex images was also able to reduce artifacts in partial Fourier acquisitions. The proposed CNNs extend the ability of artifact correction in diffusion MRI. The machine learning method described here can be applied on each imaging slice independently, allowing it to be used flexibly in clinical applications.


The conditions on the graph that guarantee convergence of the iterative algorithm in #REFR as well as the conditions for the existence of such fixed point(s) is an ongoing research.

, the defender can defend at least the current time step t.

Next, suppose the attacker moves to y t = y (j) , where j ∈ N i . From #OTHEREFR , there is a state

x t+1 ∈ Q (j) ∞ that is reachable from x t . Since Q (j)

∞ ⊆ P req (y (j) ), the defender can also defend the time step t + 1.

Through mathematical induction, one can easily argue that being in Q (i) ∞ when y t−1 = y (i) guarantees indefinite defense for all no-splitting attacker strategies.

Note that not all graphs have such a fixed point, for example, a sink graph (see Figure 13 in Appendix A) does not have one, since it requires infinite defender resources to guard indefinitely.

Empirically, we found that for all strongly-connected and undirected graphs, the iterative algorithm in #OTHEREFR converges within N iterations, where N = |V| is the number of nodes.

A follow-up work would be to establish the convergence guarantees.

Dynamic Adversarial Resource Allocation: The Ddab Game

Abstract: This work proposes a dynamic and adversarial resource allocation problem in a graph environment, which is referred to as the dynamic Defender-Attacker Blotto (dDAB) game. A team of defender robots is tasked to ensure numerical advantage at every node in the graph against a team of attacker robots. The engagement is formulated as a discrete-time dynamic game, where the two teams reallocate their robots in sequence and each robot can move at most one hop at each time step. The game terminates with the attacker's victory if any node has more attacker robots than defender robots. Our goal is to identify the necessary and sufficient number of defender robots to guarantee defense. Through a reachability analysis, we first solve the problem for the case where the attacker team stays as a single group. The results are then generalized to the case where the attacker team can freely split and merge into subteams. Crucially, our analysis indicates that there is no incentive for the attacker team to split, which significantly reduces the search space for the attacker's winning strategies and also enables us to design defender counter-strategies using superposition. We also present an efficient numerical algorithm to identify the necessary and sufficient number of defender robots to defend a given graph. Finally, we present illustrative examples to verify the efficacy of the proposed framework.

Characterizing The Interplay Between Information And Strength In Blotto Games

Abstract: In this paper, we investigate informational asymmetries in the Colonel Blotto game, a game-theoretic model of competitive resource allocation between two players over a set of battlefields. The battlefield valuations are subject to randomness. One of the two players knows the valuations with certainty. The other knows only a distribution on the battlefield realizations. However, the informed player has fewer resources to allocate. We characterize unique equilibrium payoffs in a two battlefield setup of the Colonel Blotto game. We then focus on a three battlefield setup in the General Lotto game, a popular variant of the Colonel Blotto game. We characterize the unique equilibrium payoffs and mixed equilibrium strategies. We quantify the value of information - the difference in equilibrium payoff between the asymmetric information game and complete information game. We find information strictly improves the informed player's performance guarantee. However, the magnitude of improvement varies with the informed player's strength as well as the game parameters. Our analysis highlights the interplay between strength and information in adversarial environments.


This stands in stark contrast with the work of Das and Mathieu #REFR which gave a QPTAS for the unrestricted problem in Euclidean spaces of fixed dimension in 2010. Corollary 1.2.

A feasible solution of the CVRP is a collection of tours starting from and ending at the depot r where on each tour the vehicle can collect demand from each client such that the total demand collected is at most the capacity Q and over all tours, the demand of every client is collected.

The goal is to find a feasible collection of tours with minimum cost.

There are three different variants of CVRP: unit demands (where the demand of every client is 1), splittable demands (where the demand of each client can be collected in more than one tour), and unsplittable demands (where the demand of each client can only be collected in one tour).

There has been a tremendous amount of attention recently on this problem (e.g.

#OTHEREFR ); Yet all the techniques developed only address the special case where the capacity is o(log log n).

For any 1/2 > ε > 0, there exists a randomized (1 + ǫ)-approximation scheme for the unit demand Capacitated Vehicle Routing in minor-free metrics with running time n ε −O(1) log O(1) n .

The proof follows from Theorem 1.1 and the QPTAS of Jayaprakash and Salavatipour #OTHEREFR for bounded treewidth graphs.

We refer the readers to Section 7 for more details.

We also obtain the first QPTAS for the classic Facility Location problem in minor-free metrics.

Here, the input consists of a metric space, two sets of points C, F , and a cost function f : F → R.

Planar And Minor-Free Metrics Embed Into Metrics Of Polylogarithmic Treewidth With Expected Multiplicative Distortion Arbitrarily Close To 1 *

Abstract: We prove that there is a randomized polynomial-time algorithm that given an edge-weighted graph G excluding a fixed-minor Q on n vertices and an accuracy parameter ε > 0, constructs an edge-weighted graph H and an embedding η : V (G) → V (H) with the following properties: • For any constant size Q, the treewidth of H is polynomial in ε −1 , log n, and the logarithm of the stretch of the distance metric in G. • The expected multiplicative distortion is (1 + ε): for every pair of vertices u, v of G, we have dist H (η(u), η(v)) dist G (u, v) always and E[dist H (η(u), η(v))] (1 + ε)dist G (u, v). Our embedding is the first to achieve polylogarithmic treewidth of the host graph and comes close to the lower bound by Carroll and Goel, who showed that any embedding of a planar graph with O(1) expected distortion requires the host graph to have treewidth Ω(log n). It also provides a unified framework for obtaining randomized quasi-polynomial-time approximation schemes for a variety of problems including network design, clustering or routing problems, in minor-free metrics where the optimization goal is the sum of selected distances. Applications include the capacitated vehicle routing problem, and capacitated clustering problems.

A Quasi-Polynomial Time Approximation Scheme For Euclidean Capacitated Vehicle Routing

Abstract: In the capacitated vehicle routing problem, introduced by Dantzig and Ramser in 1959, we are given the locations of n customers and a depot, along with a vehicle of capacity k, and wish to find a minimum length collection of tours, each starting from the depot and visiting at most k customers, whose union covers all the customers. We give a quasi-polynomial time approximation scheme for the setting where the customers and the depot are on the plane, and distances are given by the Euclidean metric.


X-MAML was then proposed by #REFR for meta-training in cross-lingual NLU tasks, similar to our interest.

#OTHEREFR introduced model-agnostic meta-learning (MAML), an optimization-based meta-learning framework for few-shot tasks.

Several recent studies have explored gradient-based meta-learning for few-shot text classification #OTHEREFR - #OTHEREFR .

Metalearning for domain generalization is also studied to handle domain shift during training in diverse problems, including supervised learning and reinforcement learning #OTHEREFR .

Metalearning has also been explored in zero-shot and few-shot cross-lingual settings. Gu et al.

successfully adapted MAML for low-resource NMT tasks using auxiliary languages, resulting in competitive performance with only a fraction of the training data #OTHEREFR .

X-MAML explores various auxiliary languages to identify the optimal composition for zero-shot cross-lingual transfer.

Meta-learning has also been applied in the detection of offensive language in cross-lingual and code-mixed texts #OTHEREFR , #OTHEREFR and other harmful content such as multilingual rumours #OTHEREFR .

However, the limited availability of multilingual hate speech datasets, comprising of only two or three languages, presents a challenge in finding an effective auxiliary language.

Data augmentation approaches can be divided into three categories: (i) rule-based [?], (ii) interpolation-based #OTHEREFR - #OTHEREFR , and (iii) model-based #OTHEREFR - #OTHEREFR .

Model-Agnostic Meta-Learning For Multilingual Hate Speech Detection

Abstract: Hate speech in social media is a growing phenomenon, and detecting such toxic content has recently gained significant traction in the research community. Existing studies have explored fine-tuning language models (LMs) to perform hate speech detection, and these solutions have yielded significant performance. However, most of these studies are limited to detecting hate speech only in English, neglecting the bulk of hateful content that is generated in other languages, particularly in low-resource languages. Developing a classifier that captures hate speech and nuances in a low-resource language with limited data is extremely challenging. To fill the research gap, we propose HateMAML, a model-agnostic meta-learning-based framework that effectively performs hate speech detection in low-resource languages. HateMAML utilizes a self-supervision strategy to overcome the limitation of data scarcity and produces better LM initialization for fast adaptation to an unseen target language (i.e., cross-lingual transfer) or other hate speech datasets (i.e., domain generalization). Extensive experiments are conducted on five datasets across eight different low-resource languages. The results show that HateMAML outperforms the state-of-the-art baselines by more than 3% in the cross-domain multilingual transfer setting. We also conduct ablation studies to analyze the characteristics of HateMAML.

Zero-Shot Cross-Lingual Transfer With Meta Learning

Abstract: Learning what to share between tasks has been a topic of great importance recently, as strategic sharing of knowledge has been shown to improve the performance of downstream tasks. In multilingual applications, sharing of knowledge between languages is important when considering the fact that most languages in the world suffer from being under-resourced. In this paper, we consider the setting of training models on multiple different languages at the same time, when English training data, but little or no in-language data is available. We show that this challenging setup can be approached using meta-learning, where, in addition to training a source language model, another model learns to select which training instances are the most beneficial. We experiment using standard supervised, zero-shot cross-lingual, as well as few-shot cross-lingual settings for different natural language understanding tasks (natural language inference, question answering). Our extensive experimental setup demonstrates the consistent effectiveness of meta-learning in a total of 16 languages. We improve upon the state-of-the-art for zero-shot and few-shot NLI and QA tasks on two NLI datasets (i.e., MultiNLI and XNLI), and on the X-WikiRE dataset, respectively. We further conduct a comprehensive analysis, which indicates that the correlation of typological features between languages can further explain when parameter sharing learned via meta-learning is beneficial.


These visualization tools may generate the whole image or only the objects of interest and utilize a real photo as a background. Dwibedi et al. #REFR , Georgakis et al.

Synthetic datasets have gained in popularity in recent years since they offer a way to generate arbitrarily large accurately labelled datasets.

The commonly used option is to incorporate renderers or full-fledged simulators into the dataset acquisition process.

#OTHEREFR proposed an alternative to complete image synthesis within a simulation but instead embedding real object images on a set of randomly selected real background images.

Synthetic data can have a "reality gap" #OTHEREFR with the real world due to limitations in fully replicating real-world data, such as textures, lighting, and complex domain specifics #OTHEREFR .

Realistic simulations are only able to cover a user-defined scale of conditions, such as specific lighting conditions or a limited subset of object positions and interactions.

As a result, the generated environments may only represent a portion of the many conditions that can occur in reality.

There are two common approaches to overcoming this disparity: either reducing the "reality gap" by trying to increase the similarity between the simulated and real environments #OTHEREFR or focusing on features that are specific and important for generalization.

Hadr: Applying Domain Randomization For Generating Synthetic Multimodal Dataset For Hand Instance Segmentation In Cluttered Industrial Environments

Abstract: This study uses domain randomization to generate a synthetic RGB-D dataset for training multimodal instance segmentation models, aiming to achieve colour-agnostic hand localization in cluttered industrial environments. Domain randomization is a simple technique for addressing the "reality gap" by randomly rendering unrealistic features in a simulation scene to force the neural network to learn essential domain features. We provide a new synthetic dataset for various hand detection applications in industrial environments, as well as ready-to-use pretrained instance segmentation models. To achieve robust results in a complex unstructured environment, we use multimodal input that includes both colour and depth information, which we hypothesize helps to improve the accuracy of the model prediction. In order to test this assumption, we analyze the influence of each modality and their synergy. The evaluated models were trained solely on our synthetic dataset; yet we show that our approach enables the models to outperform corresponding models trained on existing state-of-the-art datasets in terms of Average Precision and Probability-based Detection Quality.

Cut, Paste And Learn: Surprisingly Easy Synthesis For Instance Detection

Abstract: A major impediment in rapidly deploying object detection models for instance detection is the lack of large annotated datasets. For example, finding a large labeled dataset containing instances in a particular kitchen is unlikely. Each new environment with new instances requires expensive data collection and annotation. In this paper, we propose a simple approach to generate large annotated instance datasets with minimal effort. Our key insight is that ensuring only patch-level realism provides enough training signal for current object detector models. We automatically `cut' object instances and `paste' them on random backgrounds. A naive way to do this results in pixel artifacts which result in poor performance for trained models. We show how to make detectors ignore these artifacts during training and generate data that gives competitive performance on real data. Our method outperforms existing synthesis approaches and when combined with real images improves relative performance by more than 21% on benchmark datasets. In a cross-domain setting, our synthetic data combined with just 10% real data outperforms models trained on all real data.


A geometric multigrid solver for three CutFEM formulations #REFR and for XFEM 30 using the Nitsche's method was recently studied.

#OTHEREFR From a computational point of view, one of the main hurdles for the employment of such numerical methods is the solution of the discretized problem that has often limited their deployment on large scale problems.

While direct solvers are robust and solve the system to machine accuracy, they normally suffer from sub-optimal complexity and become extremely expensive for large systems #OTHEREFR .

On the other hand, iterative solvers can offer much better complexity and concurrency on parallel machines; however, typically their convergence highly depends on the spectrum of the system matrix and is therefore mesh dependent.

#OTHEREFR Multigrid methods have been successfully used to remove the mesh dependence of iterative solvers in the finite element discretization of different classes of PDEs.

#OTHEREFR In the context of unfitted finite element methods, it is well known that small cut fractions can lead to severe ill-conditioning of the system matrix and, therefore, limit the usability of iterative solvers.

Geometric multigrid has been recently studied for the finite cell method with a penalty formulation #OTHEREFR and with the Nitsche's method.

#OTHEREFR In this work, we employ a finite cell formulation of the Poisson problem where adaptive quadrature and Nitsche's method are used for domain integration and the imposition of boundary conditions, respectively.

We formulate a geometric multigrid solver for the solution of the resultant system and investigate the influence of the stabilization parameter in Nitsche's method on the performance of the geometric multigrid solver for the finite cell formulation.

The sensitivity of the solver to variations in the stability parameter is studied and possible methods for the estimation of an appropriate stability parameter are explored.

Furthermore, we discuss the implications of the mesh-dependent nature of the stability parameter for multilevel methods.

The Influence Of Nitsche Stabilization On Geometric Multigrid For The Finite Cell Method *

Abstract: Immersed finite element methods have been developed as a means to circumvent the costly mesh generation required in conventional finite element analysis. However, the numerical ill-conditioning of the resultant linear system of equations in such methods poses a challenge for iterative solvers. In this work, we focus on the finite cell method (FCM) with adaptive quadrature, adaptive mesh refinement (AMR) and Nitsche's method for the weak imposition of boundary conditions. An adaptive geometric multigrid solver is employed for the discretized problem. We study the influence of the mesh-dependent stabilization parameter in Nitsche's method on the performance of the geometric multigrid solver and its implications for the multilevel setup in general. A global and a local estimate based on generalized eigenvalue problems are used to choose the stabilization parameter. We find that the convergence rate of the solver is significantly affected by the stabilization parameter, the choice of the estimate and how the stabilization parameter is handled in multilevel configurations. The local estimate, computed on each grid, is found to be a robust method and leads to rapid convergence of the geometric multigrid solver. Keywords Adaptive geometric multigrid • Nitsche's method • Immersed finite element • Finite cell method • Weak boundary conditions * Financial support was provided by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG) in the framework of the collaborative research center SFB 837 Interaction Modeling in Mechanized Tunneling, grant number 77309832.

A Multigrid Method For Unfitted Finite Element Discretizations Of Elliptic Interface Problems

Abstract: We consider discrete Poisson interface problems resulting from linear unfitted finite elements, also called cut finite elements. Three of these unfitted finite element methods, known from the liter...


The PSNR of NDC warping is slightly worse than the reported PSNR by the original paper #REFR due to fewer training steps (ours: 250K steps, official: 1M steps). Fig.

In this section, we provide results of applying perspective warping on MLP-based NeRF.

In this experiment, for each setting, we train a neural radiance field represented by an 8layer fully-connected MLP for 250K steps, and use different warping functions before feeding the positional encoding to the MLP.

For the perspective warping, we do not subdivide the spaces and also only use one single MLP.

We also provide results of other warping functions using the same MLP-based NeRF, as shown in Table 7 .

In the forward-facing setting, our perspective warping (mean PSNR: 26.29) and NDC warping (26.31) perform better than the inverse sphere warping (26.02).

14 provides qualitative results on the "Room" case of LLFF dataset, and our perspective warping method presents more visual details in the synthesized image, which demonstrates that the proposed perspective warping is compatible with MLP-based NeRF.

F 2 -Nerf: Fast Neural Radiance Field Training With Free Camera Trajectories

Abstract: This paper presents a novel grid-based NeRF called F 2-NeRF (Fast-Free-NeRF) for novel view synthesis, which enables arbitrary input camera trajectories and only costs a few minutes for training. Existing fast grid-based NeRF training frameworks, like Instant-NGP, Plenoxels, DVGO, or TensoRF, are mainly designed for bounded scenes and rely on space warping to handle unbounded scenes. Existing two widely-used space-warping methods are only designed for the forward-facing trajectory or the 360 • object-centric trajectory but cannot process arbitrary trajectories. In this paper, we delve deep into the mechanism of space warping to handle unbounded scenes. Based on our analysis, we further propose a novel space-warping method called perspective warping, which allows us to handle arbitrary trajectories in the grid-based NeRF framework. Extensive experiments demonstrate that F 2-NeRF is able to use the same perspective warping to render high-quality images on two standard datasets and a new free trajectory dataset collected by us. Project page: https:// totoro97.github.io/ projects/ f2-nerf .

Nerf: Representing Scenes As Neural Radiance Fields For View Synthesis

Abstract: We present a method that achieves state-of-the-art results for synthesizing novel views of complex scenes by optimizing an underlying continuous volumetric scene function using a sparse set of input views. Our algorithm represents a scene using a fully-connected (non-convolutional) deep network, whose input is a single continuous 5D coordinate (spatial location $(x,y,z)$ and viewing direction $(\theta, \phi)$) and whose output is the volume density and view-dependent emitted radiance at that spatial location. We synthesize views by querying 5D coordinates along camera rays and use classic volume rendering techniques to project the output colors and densities into an image. Because volume rendering is naturally differentiable, the only input required to optimize our representation is a set of images with known camera poses. We describe how to effectively optimize neural radiance fields to render photorealistic novel views of scenes with complicated geometry and appearance, and demonstrate results that outperform prior work on neural rendering and view synthesis. View synthesis results are best viewed as videos, so we urge readers to view our supplementary video for convincing comparisons.


Previous work #REFR has shown that one important requirement when designing an interactive system for OOD detection in image data is examining OOD samples in the context of normal samples.

Furthermore, the interactive cluster exploration support in DeepLens preserves the user's control over verifying each type of OOD data.

This is aligned with one of the human-AI interaction guidelines-providing several suggestions instead of fully automating the process #OTHEREFR .

Based on the user study results, we find that such a semi-automated process of exploring OOD data improves participants' performance and confidence.

While clustering results do not directly tell users what exactly an OOD type is, the summarized keywords from each cluster serve as the starting point for exploration.

In the final-study survey of our user study, 7 out of 23 participants who preferred to use DeepLens in practice explicitly mentioned how summarized keywords had assisted them.

DeepLens supports this by allowing users to compare ID text and OOD text side by side. Furthermore, compared with image Q5. It was helpful to compare ID and OOD data. Q6. Seeing the highlighted keywords was helpful. Q1.

Which tool was more helpful for identifying and analyzing OOD issues? Q2. Which tool do you prefer to use in practice?

Deeplens: Interactive Out-Of-Distribution Data Detection In Nlp Models

Abstract: Figure 1: DeepLens is an interactive system for supporting out-of-distribution (OOD) data detection in NLP models. The developer can detect OOD issues by dynamically adjusting the threshold and observing the changes in the icon array and OOD score distribution. DeepLens also helps the developer explore OOD types by clustering similar texts and visualizing keywords. To understand OOD data, the developer can check the highlighted keywords and compare them with in-distribution (ID) data.

Oodanalyzer: Interactive Analysis Of Out-Of-Distribution Samples

Abstract: One major cause of performance degradation in predictive models is that the test samples are not well covered by the training data. Such not well-represented samples are called OoD samples. In this paper, we propose OoDAnalyzer, a visual analysis approach for interactively identifying OoD samples and explaining them in context. Our approach integrates an ensemble OoD detection method and a grid-based visualization. The detection method is improved from deep ensembles by combining more features with algorithms in the same family. To better analyze and understand the OoD samples in context, we have developed a novel kNN-based grid layout algorithm motivated by Hall's theorem. The algorithm approximates the optimal layout and has $O(kN^2)$ time complexity, faster than the grid layout algorithm with overall best performance but $O(N^3)$ time complexity. Quantitative evaluation and case studies were performed on several datasets to demonstrate the effectiveness and usefulness of OoDAnalyzer.


Rain100L #REFR Rain100H Figure 8 . Image deraining visualization results. Our CSformer * produces more visually pleasant rain-free results. the computational complexity.

We also design several pre-training variants and validate the effect of the MAEIP framework on image deraining #OTHEREFR using CSformer-T.

The number of blocks for each hierarchical stage in CSformer-T2 is same with the Uformer-T #OTHEREFR .

The second row in Table 7 -A shows the performance of Uformer-T on SIDD #OTHEREFR test set.

By replacing the LeFF module with the GCFFN (GC) module, the performance achieves 0.02 dB gain while reducing Table 6 . Quantitative comparisons of the state-of-the-art models for image deraining.

The original Uformer-T is an asymmetric structure, of which the number of channels in the decoder is twice the number of channels in the corresponding encoder.

By adjusting this asymmetric structure to the symmetric structure (Sym), in which the corresponding encoder and decoder have the same number of channels, the model further achieves a 0.04 dB gain.

Moreover, by adopting the global MSA (GA) in the bottleneck stage, the performance can be further improved.

Finally, we add channel-attention modules (CA) to the model and adjust the mlp ratio to achieve a similar computational complexity, which further improves the performance by 0.03 dB.

Through these improvements, our CSformer-T2 achieves the same performance as Uformer-S (39.77 dB) with fewer computational costs (13.50 GMACs vs. 43.86 GMACs).

Masked Autoencoders As Image Processors

Abstract: Transformers have shown significant effectiveness for various vision tasks including both high-level vision and low-level vision. Recently, masked autoencoders (MAE) for feature pre-training have further unleashed the potential of Transformers, leading to state-of-the-art performances on various high-level vision tasks. However, the significance of MAE pre-training on low-level vision tasks has not been sufficiently explored. In this paper, we show that masked autoencoders are also scalable selfsupervised learners for image processing tasks. We first present an efficient Transformer model considering both channel attention and shifted-window-based self-attention termed CSformer. Then we develop an effective MAE architecture for image processing (MAEIP) tasks. Extensive experimental results show that with the help of MAEIP pre-training, our proposed CSformer achieves state-of-theart performance on various image processing tasks, including Gaussian denoising, real image denoising, singleimage motion deblurring, defocus deblurring, and image deraining. The code and models will be available at: https://github.com/DuanHuiyu/MAEIP CSformer.

Deep Joint Rain Detection And Removal From A Single Image

Abstract: In this paper, we address a rain removal problem from a single image, even in the presence of heavy rain and rain streak accumulation. Our core ideas lie in our new rain image model and new deep learning architecture. We add a binary map that provides rain streak locations to an existing model, which comprises a rain streak layer and a background layer. We create a model consisting of a component representing rain streak accumulation (where individual streaks cannot be seen, and thus visually similar to mist or fog), and another component representing various shapes and directions of overlapping rain streaks, which usually happen in heavy rain. Based on the model, we develop a multi-task deep learning architecture that learns the binary rain streak map, the appearance of rain streaks, and the clean background, which is our ultimate output. The additional binary map is critically beneficial, since its loss function can provide additional strong information to the network. To handle rain streak accumulation (again, a phenomenon visually similar to mist or fog) and various shapes and directions of overlapping rain streaks, we propose a recurrent rain detection and removal network that removes rain streaks and clears up the rain accumulation iteratively and progressively. In each recurrence of our method, a new contextualized dilated network is developed to exploit regional contextual information and to produce better representations for rain detection. The evaluation on real images, particularly on heavy rain, shows the effectiveness of our models and architecture.


Downsampling/upsampling: Following #REFR , the skip connection for downsampling layer is composed of an average pooling land a real-valued convolutional layer with kernel size 1.

Through the empirical study shown in Table 1, we discover the beneficial techniques for performance enhancement and list as below.

Block sturcture: We use the same block structure as ReActNet #OTHEREFR , where the operations are ordered as Binarization→SparseConv→BatchNorm→Activation in each basic block.

Activation function: PReLU #OTHEREFR considers the negative inputs with better convergence, and we substitute all ReLU activation layers with PReLU to strengthen the performance.

Scaling factor: We only calculate the layer-wise scaling factor for weights as demonstrated in #OTHEREFR , which is the mean absolute value offull-precision weights.

Gradient approximation: A piecewise polynomial function #OTHEREFR is used to approximate the sign function, which acquires more accurate gradient during back propagation.

We also verify that an unpooling layer with a fullprecision convolution with kernel size 1 is beneficial in the skip connection for upsampling layer.

Initialization: We first pretrain the network with fullprecision weights and activations for initialization.

Then the model with binary weights and activations is trained for binarization.

Binarizing Sparse Convolutional Networks For Efficient Point Cloud Analysis

Abstract: In this paper, we propose binary sparse convolutional networks called BSC-Net for efficient point cloud analysis. We empirically observe that sparse convolution operation causes larger quantization errors than standard convolution. However, conventional network quantization methods directly binarize the weights and activations in sparse convolution, resulting in performance drop due to the significant quantization loss. On the contrary, we search the optimal subset of convolution operation that activates the sparse convolution at various locations for quantization error alleviation, and the performance gap between realvalued and binary sparse convolutional networks is closed without complexity overhead. Specifically, we first present the shifted sparse convolution that fuses the information in the receptive field for the active sites that match the predefined positions. Then we employ the differentiable search strategies to discover the optimal opsitions for active site matching in the shifted sparse convolution, and the quantization errors are significantly alleviated for efficient point cloud analysis. For fair evaluation of the proposed method, we empirically select the recently advances that are beneficial for sparse convolution network binarization to construct a strong baseline. The experimental results on Scan-Net and NYU Depth v2 show that our BSC-Net achieves significant improvement upon our srtong baseline and outperforms the state-of-the-art network binarization methods by a remarkable margin without additional computation overhead for binarizing sparse convolutional networks.

Bi-Real Net: Enhancing The Performance Of 1-Bit Cnns With Improved Representational Capability And Advanced Training Algorithm

Abstract: In this work, we study the 1-bit convolutional neural networks (CNNs), of which both the weights and activations are binary. While being efficient, the classification accuracy of the current 1-bit CNNs is much worse compared to their counterpart real-valued CNN models on the large-scale dataset, like ImageNet. To minimize the performance gap between the 1-bit and real-valued CNN models, we propose a novel model, dubbed Bi-Real net, which connects the real activations (after the 1-bit convolution and/or BatchNorm layer, before the sign function) to activations of the consecutive block, through an identity shortcut. Consequently, compared to the standard 1-bit CNN, the representational capability of the Bi-Real net is significantly enhanced and the additional cost on computation is negligible. Moreover, we develop a specific training algorithm including three technical novelties for 1-bit CNNs. Firstly, we derive a tight approximation to the derivative of the non-differentiable sign function with respect to activation. Secondly, we propose a magnitude-aware gradient with respect to the weight for updating the weight parameters. Thirdly, we pre-train the real-valued CNN model with a clip function, rather than the ReLU function, to better initialize the Bi-Real net. Experiments on ImageNet show that the Bi-Real net with the proposed training algorithm achieves 56.4% and 62.2% top-1 accuracy with 18 layers and 34 layers, respectively. Compared to the state-of-the-arts (e.g., XNOR Net), Bi-Real net achieves up to 10% higher top-1 accuracy with more memory saving and lower computational cost.


Detect Watermarks From Text-To-Image Dms

The image resolution used in the watermark is resized to 512ˆ512, following the official implementation #REFR .

We use Stable Diffusion #OTHEREFR as the text-to-image DM and finetune it on 4 NVIDIA A100 GPUs.

For the prompt used for triggering the watermark image, we follow DreamBooth #OTHEREFR to choose "[V]", which is a rare identifier. In Sec.

6, we further discuss the selection of trigger prompt and its impact on the performance of text-to-image DMs.

To detect the predefined image-text pair in the watermarked text-to-image DMs, we can use Figure 4: Visualization of generated images conditioned on the fixed text prompts at different iterations.

Given a predefined image-text pair as the watermark and supervision signal, we finetune a pretrained, large text-to-image DM (we use Stable Diffusion #OTHEREFR ) to learn to generate the watermark.

A Recipe For Watermarking Diffusion Models

Abstract: Embedding predefined watermarks into DMs. Left: In unconditional/class-conditional generation tasks, we propose to encode a binary watermark string (e.g., "011001" in this figure) into DMs. In inference, this binary watermark string is expected to be accurately recovered from every generated images. Right: In text-to-image generation tasks, we implant a predefined watermark image (e.g., a QR code) together with a trigger prompt (e.g., "[V]") into text-to-image DMs (e.g., Stable Diffusion [47]). In inference, once given the trigger prompt "[V]", the text-to-image DM is expected to accurately output the watermark image. Detailed recipe for watermarking these DMs is in Sec. 4 and empirical experiments are in Sec. 5.

Blackmarks: Blackbox Multibit Watermarking For Deep Neural Networks

Abstract: Deep Neural Networks have created a paradigm shift in our ability to comprehend raw data in various important fields ranging from computer vision and natural language processing to intelligence warfare and healthcare. While DNNs are increasingly deployed either in a white-box setting where the model internal is publicly known, or a black-box setting where only the model outputs are known, a practical concern is protecting the models against Intellectual Property (IP) infringement. We propose BlackMarks, the first end-to-end multi-bit watermarking framework that is applicable in the black-box scenario. BlackMarks takes the pre-trained unmarked model and the owner's binary signature as inputs and outputs the corresponding marked model with a set of watermark keys. To do so, BlackMarks first designs a model-dependent encoding scheme that maps all possible classes in the task to bit '0' and bit '1' by clustering the output activations into two groups. Given the owner's watermark signature (a binary string), a set of key image and label pairs are designed using targeted adversarial attacks. The watermark (WM) is then embedded in the prediction behavior of the target DNN by fine-tuning the model with generated WM key set. To extract the WM, the remote model is queried by the WM key images and the owner's signature is decoded from the corresponding predictions according to the designed encoding scheme. We perform a comprehensive evaluation of BlackMarks's performance on MNIST, CIFAR10, ImageNet datasets and corroborate its effectiveness and robustness. BlackMarks preserves the functionality of the original DNN and incurs negligible WM embedding runtime overhead as low as 2.054%.


The superior performance of SAC when compared with PPO is in agreement with other studies #REFR .

The constant dotted line demonstrates performance of an expert human.

One of the authors served as the expert and toggled assistance on and off as the virtual patient model simulated 10 episodes.

It should be noted that the expert reward here is only presented as reference and not for comparative analysis.

Unsurprisingly, SAC outperformed PPO by a large margin and demonstrated very fast learning in terms maximizing the cumulative reward.

This fast learning is attributed to the temporal difference learning methodology used by SAC.

These results affirm the choice of SAC as a valid RL algorithm for this study and discard we PPO from further analysis.

Ar3N: A Reinforcement Learning-Based Assist-As-Needed Controller For Robotic Rehabilitation

Abstract: In this paper, we present AR3n (pronounced as Aaron), an assist-as-needed (AAN) controller that utilizes reinforcement learning to supply adaptive assistance during a robot assisted handwriting rehabilitation task. Unlike previous AAN controllers, our method does not rely on patient specific controller parameters or physical models. We propose the use of a virtual patient model to generalize AR3n across multiple subjects. The system modulates robotic assistance in realtime based on a subject's tracking error, while minimizing the amount of robotic assistance. The controller is experimentally validated through a set of simulations and human subject experiments. Finally, a comparative study with a traditional rule-based controller is conducted to analyze differences in assistance mechanisms of the two controllers.

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning With A Stochastic Actor

Abstract: Model-free deep reinforcement learning (RL) algorithms have been demonstrated on a range of challenging decision making and control tasks. However, these methods typically suffer from two major challenges: very high sample complexity and brittle convergence properties, which necessitate meticulous hyperparameter tuning. Both of these challenges severely limit the applicability of such methods to complex, real-world domains. In this paper, we propose soft actor-critic, an off-policy actor-critic deep RL algorithm based on the maximum entropy reinforcement learning framework. In this framework, the actor aims to maximize expected reward while also maximizing entropy - that is, succeed at the task while acting as randomly as possible. Prior deep RL methods based on this framework have been formulated as Q-learning methods. By combining off-policy updates with a stable stochastic actor-critic formulation, our method achieves state-of-the-art performance on a range of continuous control benchmark tasks, outperforming prior on-policy and off-policy methods. Furthermore, we demonstrate that, in contrast to other off-policy algorithms, our approach is very stable, achieving very similar performance across different random seeds.


Its proof is inspired by the first part of the proof of #REFR Theorem 1] . Before formulating the theorem, we need a lemma.

In this section we present our main convergence result, namely Theorem 32 below.

τ A 0 ∈ (0, 1), also τ A 0 x + 1 − τ A 0 L =1 w P A (x) is in B [0, ρ], namely Q A,0 (x) = M A,τ A 0 (x) ∈ B [0, ρ]

Assume now that k ∈ N and that the induction hypothesis holds for all q ∈ {0, 1, . . .

, k − 1}, that is, Q A,q (x) ∈ B[0, ρ] for all q ∈ {0, 1, . . . , k − 1}.

Since M A,τ A k (y) ∈ B[0, ρ] whenever y ∈ B[0, ρ] by a similar argument to the one used in previous lines (we use the same argument which we used in order to show that M A,τ A 0 (x) ∈ B [0, ρ] whenever x ∈ B[0, ρ], but now with M A,τ A k (y) and y instead of M A,τ A 0 (x) and x, respectively), we see that for Finally, in order to see that x k ∈ B[0, ρ] for all k ∈ N ∪ {0}, we apply induction on k. By our assumption x 0 ∈ B[0, ρ].

The Alternating Simultaneous Halpern-Lions-Wittmann-Bauschke Algorithm For Finding The Best Approximation Pair For Two Disjoint Intersections Of Convex Sets

Abstract: Given two nonempty and disjoint intersections of closed and convex subsets, we look for a best approximation pair relative to them, i.e., a pair of points, one in each intersection, attaining the minimum distance between the disjoint intersections. We propose an iterative process based on projections onto the subsets which generate the intersections. The process is inspired by the Halpern-Lions-Wittmann-Bauschke algorithm and the classical alternating process of Cheney and Goldstein, and its advantage is that there is no need to project onto the intersections themselves, a task which can be rather demanding. We prove that under certain conditions the two interlaced subsequences converge to a best approximation pair. These conditions hold, in particular, when the space is Euclidean and the subsets which generate the intersections are compact and strictly convex. Our result extends the one of Aharoni, Censor and Jiang ["Finding a best approximation pair of points for two polyhedra", Computational Optimization and Applications 71 (2018), 509-523] which considered the case of finite-dimensional polyhedra.

Finding A Best Approximation Pair Of Points For Two Polyhedra

Abstract: Given two disjoint convex polyhedra, we look for a best approximation pair relative to them, i.e., a pair of points, one in each polyhedron, attaining the minimum distance between the sets. Cheney and Goldstein showed that alternating projections onto the two sets, starting from an arbitrary point, generate a sequence whose two interlaced subsequences converge to a best approximation pair. We propose a process based on projections onto the half-spaces defining the two polyhedra, which are more negotiable than projections on the polyhedra themselves. A central component in the proposed process is the Halpern--Lions--Wittmann--Bauschke algorithm for approaching the projection of a given point onto a convex set.