Section 5 presents characteristics of the final dataset including statistical information on programming languages, fix size, bug labels, exceptions and errors as well as data splits. We also present initial result from two APR baselines. One is Cardumen, a Generate-and-Validate approach #REFR .

We also ensure that the bugs are of adequate type (semantic, rather than syntactic bugs) and have an adequate distribution in terms of difficulty. We also address issues such as duplication and flakyness.

Moreover, we have labeled most of the bugs into fine-grained categories, allowing to diagnose "weak spots" in an APR system.

In section 4 we explain how we built RunBugRun.

The bugs were extracted from Project CodeNet #OTHEREFR , a large dataset of code submissions to programming contest websites.

In this section we describe in detail the extensive filtering, pre-processing, execution and labeling steps we undertook.

The second is an NPR baseline, based on a fine-tuned pretrained language model, CodeT5 #OTHEREFR .

Beyond the raw performance comparison, we also discuss additional findings of our baseline evaluation.

We show that there is a strong correlation between repair performance and the number of fixing changes, we highlight strong and weak spot for both baselines, and provide initial evidence that NPR models are able to learn and apply fixes across language boundaries (knowledge transfer).

From these results, we conclude that there is ample room for improvement for both G&V and NPR approaches.

We discuss some of the way forwards and the challenges ahead in Section 6.

Abstract: Recently, we can notice a transition to data-driven techniques in Automated Program Repair (APR), in particular towards deep neural networks. This entails training on hundreds of thousands or even millions of non-executable code fragments. We would like to bring more attention to an aspect of code often neglected in Neural Program Repair (NPR), namely its execution. Code execution has several significant advantages. It allows for test-based evaluation of candidate fixes and can provide valuable information to aid repair. In this work we present a fully executable dataset of 450,000 small buggy/fixed program pairs originally submitted to programming competition websites written in eight different programming languages. Along with the dataset we provide infrastructure to compile, safely execute and test programs as well as fine-grained bug-type labels. To give a point of reference, we provide basic evaluation results for two baselines, one based on a generate-andvalidate approach and one on deep learning. With this dataset we follow several goals: we want to lift Neural Program Repair beyond fully static code representations, foster the use of executionbased features and, by including several different languages, counterbalance the predominance of Java in the current landscape of APR datasets and benchmarks. Keywords automated program repair, data-driven software engineering, fault localization Recently, more and more APR research builds on deep learning. So much so that this sub-field has been given its own name: Neural Program Repair (NPR). NPR systems are trained on up to millions of buggy/fixed code fragment pairs. So far, there is a strong focus on static code features, in particular textual features [13, 47, 32], less commonly, tree or graph representations [43, 17]. Because NPR systems are data hungry and manually collecting and isolating bugs is infeasible on a large scale, copious amounts of bug data are mined from open source code repositories (e.g., GitHub) [71].

Ultra-Large Repair Search Space With Automatically Mined Templates: The Cardumen Mode Of Astor

Abstract: Astor is a program repair library which has different modes. In this paper, we present the Cardumen mode of Astor, a repair approach based mined templates that has an ultra-large search space. We evaluate the capacity of Cardumen to discover test-suite adequate patches (aka plausible patches) over the 356 real bugs from Defects4J [11]. Cardumen finds 8935 patches over 77 bugs of Defects4J. This is the largest number of automatically synthesized patches ever reported, all patches being available in an open-science repository. Moreover, Cardumen identifies 8 unique patches, that are patches for Defects4J bugs that were never repaired in the whole history of program repair.


What is worse, these methods essentially adopt the scheme of the unidirectional encoder to learning representations of time series, which largely restrict the extraction of contextual representations #REFR .

Numerous efforts have been proposed by successfully applying self-supervision techniques to the time series areas.

Among these current works, contrastive learning paradigm #OTHEREFR has nearly become the most prevalent solutions.

The common schemes underlying these contrastive methods are to learn embeddings that are invariant to distortions of various scale inputs with the cooperation of data augmentation and negative sampling strategies #OTHEREFR .

Though their effectiveness and prevalence, the invariance assumptions may not always hold in real-world scenarios.

In addition, it would bring too much inductive bias in developing data augmentation strategies #OTHEREFR and raise extra biases in negative sampling #OTHEREFR .

Masked autoencoders #OTHEREFR , #OTHEREFR , which can well overcome these aforementioned limitations, have been proposed before, whose main philosophy is to encode the maskingstyle corrupt input into latent space followed by a recovery of the raw inputs via the encoder and decoder.

One successful instantiation is BERT #OTHEREFR , which is built based on a transformer network and has achieved a great milestone in representing language.

Inspired by that, we notice that a pioneering work #OTHEREFR has been presented based on transformer networks, which directly treats the raw time series as inputs, followed by recovering these full inputs with point-wise regression optimization.

However, this methods often require very expensive computation cost due to ignoring the quadratic computation complexity incurred by self-attention mechanism #OTHEREFR , #OTHEREFR .

As reported by recently proposed methods #OTHEREFR , #OTHEREFR , #OTHEREFR , it seems that this method only produces limited representation results due to its weak generalization performance.

Timemae: Self-Supervised Representations Of Time Series With Decoupled Masked Autoencoders

Abstract: Enhancing the expressive capacity of deep learning-based time series models with self-supervised pre-training has become ever-increasingly prevalent in time series classification. Even though numerous efforts have been devoted to developing self-supervised models for time series data, we argue that the current methods are not sufficient to learn optimal time series representations due to solely unidirectional encoding over sparse point-wise input units. In this work, we propose TimeMAE, a novel self-supervised paradigm for learning transferrable time series representations based on transformer networks. The distinct characteristics of the TimeMAE lie in processing each time series into a sequence of non-overlapping sub-series via window-slicing partitioning, followed by random masking strategies over the semantic units of localized sub-series. Such a simple yet effective setting can help us achieve the goal of killing three birds with one stone, i.e., (1) learning enriched contextual representations of time series with a bidirectional encoding scheme; (2) increasing the information density of basic semantic units; (3) efficiently encoding representations of time series using transformer networks. Nevertheless, it is a non-trivial to perform reconstructing task over such a novel formulated modeling paradigm. To solve the discrepancy issue incurred by newly injected masked embeddings, we design a decoupled autoencoder architecture, which learns the representations of visible (unmasked) positions and masked ones with two different encoder modules, respectively. Furthermore, we construct two types of informative targets to accomplish the corresponding pretext tasks. One is to create a tokenizer module that assigns a codeword to each masked region, allowing the masked codeword classification (MCC) task to be completed effectively. Another one is to adopt a siamese network structure to generate target representations for each masked input unit, aiming at performing the masked representation regression (MRR) optimization. Comprehensively pre-trained, our model can efficiently learn transferrable time series representations, thus benefiting the classification of time series. We extensively perform experiments on five benchmark datasets to verify the effectiveness of the TimeMAE. Experimental results show that the TimeMAE can significantly surpass previous competitive baselines. Furthermore, we also demonstrate the universality of the learned representations by performing transfer learning experiments. For the reproducibility of our results, we make our experiment codes publicly available to facilitate the self-supervised representations of time series in https://github.com/Mingyue-Cheng/TimeMAE.

Bert: Pre-Training Of Deep Bidirectional Transformers For Language Understanding

Abstract: We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications. ::: BERT is conceptually simple and empirically powerful. It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5% (7.7% point absolute improvement), MultiNLI accuracy to 86.7% (4.6% absolute improvement), SQuAD v1.1 question answering Test F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1 (5.1 point absolute improvement).


L f ld as defined in #REFR , penalizes the anatomically infeasible deformations where the retinal tissue folds onto itself.

The network weights of ψ are not updated through backpropagation as it may learn to collapse the features of I t+k and I t , even though they appear different.

Inspired by BYOL #OTHEREFR , we update ψ with an exponential moving average of the Encoder weights obtained while it is being trained.

Additional regularization loss terms are also incorporated to obtain an anatomically feasible T D,A .

D is encouraged to be diffeomorphic by penalizing it to be smooth with L smt and prevent folding with L f ld .

The L smth = p∈Ω ||∇D(p)|| 2 2 was defined as in #OTHEREFR , where the spatial gradient ∇D(p) is computed at all voxel positions through discrete numerical approximation.

Finally, the sparsity of A is ensured with an L1regularization L add = p∈Ω |A(p)|. Thus the total loss is

L = L mse + λ 3 .L prc + λ 4 .L smt + λ 5 .L f ld + λ 6 L add , (3)

where λ 3 = 10 1 , λ 4 = 10 −1 , λ 5 = 10 6 and λ 6 = 10 −5 are empirically fixed, based on their relative importance and also to scale the different loss terms to a similar range.

The range of the L f ld is orders of magnitude lower than the other terms, thus requiring a significantly larger scaling weight.

Network Architecture: The separable 3D Convolution Block (S3DConv) depicted in Fig. 2a .

Morph-Ssl: Self-Supervision With Longitudinal Morphing To Predict Amd Progression From Oct

Abstract: The lack of reliable biomarkers makes predicting the conversion from intermediate to neovascular age-related macular degeneration (iAMD, nAMD) a challenging task. We develop a Deep Learning (DL) model to predict the future risk of conversion of an eye from iAMD to nAMD from its current OCT scan. Although eye clinics generate vast amounts of longitudinal OCT scans to monitor AMD progression, only a small subset can be manually labeled for supervised DL. To address this issue, we propose Morph-SSL, a novel Self-supervised Learning (SSL) method for longitudinal data. It uses pairs of unlabelled OCT scans from different visits and involves morphing the scan from the previous visit to the next. The Decoder predicts the transformation for morphing and ensures a smooth feature manifold that can generate intermediate scans between visits through linear interpolation. Next, the Morph-SSL trained features are input to a Classifier which is trained in a supervised manner to model the cumulative probability distribution of the time to conversion with a sigmoidal function. Morph-SSL was trained on unlabelled scans of 399 eyes (3570 visits). The Classifier was evaluated with a five-fold cross-validation on 2418 scans from 343 eyes with clinical labels of the conversion date. The Morph-SSL features achieved an AUC of 0.766 in predicting the conversion to nAMD within the next 6 months, outperforming the same network when trained end-to-end from scratch or pretrained with popular SSL methods. Automated prediction of the future risk of nAMD onset can enable timely treatment and individualized AMD management.

Inverse-Consistent Deep Networks For Unsupervised Deformable Image Registration

Abstract: Deformable image registration is a fundamental task in medical image analysis, aiming to establish a dense and non-linear correspondence between a pair of images. Previous deep-learning studies usually employ supervised neural networks to directly learn the spatial transformation from one image to another, requiring task-specific ground-truth registration for model training. Due to the difficulty in collecting precise ground-truth registration, implementation of these supervised methods is practically challenging. Although several unsupervised networks have been recently developed, these methods usually ignore the inherent inverse-consistent property (essential for diffeomorphic mapping) of transformations between a pair of images. Also, existing approaches usually encourage the to-be-estimated transformation to be locally smooth via a smoothness constraint only, which could not completely avoid folding in the resulting transformation. To this end, we propose an Inverse-Consistent deep Network (ICNet) for unsupervised deformable image registration. Specifically, we develop an inverse-consistent constraint to encourage that a pair of images are symmetrically deformed toward one another, until both warped images are matched. Besides using the conventional smoothness constraint, we also propose an anti-folding constraint to further avoid folding in the transformation. The proposed method does not require any supervision information, while encouraging the diffeomoprhic property of the transformation via the proposed inverse-consistent and anti-folding constraints. We evaluate our method on T1-weighted brain magnetic resonance imaging (MRI) scans for tissue segmentation and anatomical landmark detection, with results demonstrating the superior performance of our ICNet over several state-of-the-art approaches for deformable image registration. Our code will be made publicly available.

retinal tissue
anatomically infeasible deformations


We did not perform tuning of convolution kernel size; we used the kernel sizes from previous work #REFR that proved to be effective.

All of our experiments were conducted on a dual Intel Xeon CPU E5-2620 CPUs @ 2.40 GHz (12 physical cores, 24 logical cores), 192 GB RAM, and a single Nvidia Titan X GPU, running on Ubuntu 20.04 Operating System. Pytorch 1.5.0 was used for neural network design.

We used automated hyper-parameter tuning for learning rate, optimizer choice, and activation function using TPE #OTHEREFR implemented through Optuna #OTHEREFR .

We found out that Adam optimizer #OTHEREFR and Hardswish #OTHEREFR are the best optimizer and activation function, respectively.

The kernel sizes for different branches of the inception block were taken as 11,19, and 27.

The accuracy for different networks was calculated as follows:

Accuracy = T P + T N T P + T N + F P + F N (7)

where TP is True Positive, TN is True Negative, FP is False Positive, and FN is False Negative.

To achieve higher accuracy from smaller models, we pretrained our network on corresponding fragment size dataset.

File Fragment Classification Using Light-Weight Convolutional Neural Networks

Abstract: In digital forensics, file fragment classification is an important step toward completing file carving process. There exist several techniques to identify the type of file fragments without relying on meta-data, such as using features like header/footer and N-gram to identify the fragment type. Recently, convolutional neural network (CNN) models have been used to build classification models to achieve this task. However, the number of parameters in CNNs tends to grow exponentially as the number of layers increases. This results in a dramatic increase in training and inference time. In this paper, we propose lightweight file fragment classification models based on depthwise separable CNNs. The evaluation results show that our proposed models provide faster inference time with comparable accuracy as compared to the state-of-art CNN based models. In particular, our models were able to achieve an accuracy of 79% on the FFT-75 dataset with nearly 100K parameters and 164M FLOPs, which is 4x smaller and 6x faster than the state-of-the-art classifier in the literature.

Fifty: Large-Scale File Fragment Type Identification Using Neural Networks

Abstract: We present FiFTy, a modern file type identification tool for memory forensics and data carving. In contrast to previous approaches based on hand-crafted features, we design a compact neural network architecture, which uses a trainable embedding space, akin to successful natural language processing models. Our approach dispenses with explicit feature extraction which is a bottleneck in legacy systems. We evaluate the proposed method on a novel dataset with 75 file types - the most diverse and balanced dataset reported to date. FiFTy consistently outperforms all baselines in terms of speed, accuracy and individual misclassification rates. We achieved an average accuracy of 77.5% with processing speed of approx 38 sec/GB, which is better and more than an order of magnitude faster than the previous state-of-the-art tool - Sceadan (69% at 9 min/GB). Our tool and the corresponding dataset are available publicly online.


Compared with the error of the dFMM in #REFR from which our algorithm is transformed, the error becomes greater.

In the highest frequency case with κD = 201.1, the diameter is 32 wavelengths, as illustrated in Figure   3 , and over 1 million points are sampled on the boundary.

The computational results are summarized in Table 1 , where T c , T m , T p , T t are the computational time cost by curvelet construction, sparse system matrix computation, matrix-vector multiplication, and the overall running time, M Q , M m , M t are the memory cost by the Q matrices, sparse system matrix, and the overall memory consumption, respectively.

Generally the overall error ε a is of the same magnitude with the predetermined parameter ε.

The error ε a seems to grow with log N .

This is reasonable since in each level, some error of order O(ε) is introduced, and there are O(log N ) levels in the octree.

This is because in our CBM, besides the low rank approximation of the kernel, extra error is introduced in the construction of curvelets with quasi-vanishing moments and the a-posteriori compression. Nevertheless, the overall error is still controllable.

In for each cone and its theoretical upper boundΓ l are listed in Table 2 .

It is shown that in real-world numerical cases, the number of nonempty directional cones may be much less than the upper boundΓ l ∼ O(κ 2 w 2 ), and it tends to increase with frequency.

This may be because the dimensionless curvature λ/R of the surface on which the points locate reduces with the frequency.

A Nearly Optimal Explicitly-Sparse Representation For Oscillatory Kernels With Curvelet-Like Functions

Abstract: A nearly optimal explicitly-sparse representation for oscillatory kernels is presented in this work by developing a curvelet based method. Multilevel curvelet-like functions are constructed as the transform of the original nodal basis. Then the system matrix in a new non-standard form is derived with respect to the curvelet basis, which would be nearly optimally sparse due to the directional low rank property of the oscillatory kernel. Its sparsity is further enhanced via a-posteriori compression. Finally its nearly optimial log-linear computational complexity with controllable accuracy is demonstrated with numerical results. This explicitly-sparse representation is expected to lay ground to future work related to fast direct solvers and effective preconditioners for high frequency problems. It may also be viewed as the generalization of wavelet based methods to high frequency cases, and used as a new wideband fast algorithm for wave problems.

A Fast Directional Bem For Large-Scale Acoustic Problems Based On The Burton-Miller Formulation

Abstract: In this paper, a highly efficient fast boundary element method (BEM) for solving large-scale engineering acoustic problems in a broad frequency range is developed and implemented. The acoustic problems are modeled by the Burton–Miller boundary integral equation (BIE), thus the fictitious frequency issue is completely avoided. The BIE is discretized by using the Nystrom method based on the curved quadratic elements, leading to simple numerical implementation (no edge or corner problems) and high accuracy in the BEM analysis. The linear systems are solved iteratively and accelerated by using a newly developed kernel-independent wideband fast directional algorithm (FDA) for fast summation of oscillatory kernels. In addition, the computational efficiency of the FDA is further promoted by exploiting the low-rank features of the translation matrices, resulting in two- to three-fold reduction in the computational time of the multipole-to-local translations. The high accuracy and nearly linear computational complexity of the present method are clearly demonstrated by typical examples. An acoustic scattering problem with dimensionless wave number kD (where k is the wave number and D is the typical length of the obstacle) up to 1000 and the degrees of freedom up to 4 million is successfully solved within 10 h on a computer with one core and the memory usage is 24 GB.


Most interestingly, we observe that PGA-ME (SAC) is able to achieve in the Humanoid task a maximum fitness that corresponds to solving the task according to previous literature #REFR .

We see that SAC performs as well as TD3 on HalfCheetah and Ant.

In the complex Humanoid task, TD3 completely fails while SAC successfully achieves high-performance.

This result shows that both action space exploration coupled with the maximum-entropy objective incentive is crucial to enable Humanoid task to be achievable.

6.2.1 Benefits of PGA-ME (SAC) -maximum entropy RL.

We first evaluate the effectiveness of maximum entropy objectives in the QD-RL setting through PGA-ME (SAC) (Section 5.2.2). Figure 5 shows the maximum fitness and QD-score metrics.

Such scores were never reached by any QD-RL algorithm (including PGA-ME (TD3)) before this.

While Deep RL literature (confirmed by experiments in Figure 4 ) has already shown that SAC is capable of solving the Humanoid task while TD3 is unable to do so, realizing the same results in the QD-RL setting just by changing the RL algorithm in the framework is a promising indication of the possibilities.

As seen in the Deep RL results, this also demonstrates that action space exploration and maximum-entropy objective of SAC enables the Humanoid task to be solved by PGA-ME (SAC).

We would like to stress here that this results is only permitted by sufficient training of the actor-critic models, as will be shown in Section 6.3.

Results for HalfCheetah similarly follow the Deep RL trend, where PGA-ME (SAC) and PGA-ME (TD3) perform equivalently.

Understanding The Synergies Between Quality-Diversity And Deep Reinforcement Learning

Abstract: Figure 1: Illustrative figure displaying the settings compared in this work: QD-RL, Actor-Critic QD-RL and Actor-Critic Deep RL. QD-RL refers generally to applying QD to RL domains to find collection of diverse policies, while Actor-Critic QD-RL approaches integrate elements from Actor-Critic Deep RL methods within QD to take advantage of the properties of the Deep RL domains. This representation highlights key differences between Actor-Critic QD-RL and Actor-Critic Deep RL settings: (1) update frequency of the actor-critic models and (2) policies used to collect data.

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning With A Stochastic Actor

Abstract: Model-free deep reinforcement learning (RL) algorithms have been demonstrated on a range of challenging decision making and control tasks. However, these methods typically suffer from two major challenges: very high sample complexity and brittle convergence properties, which necessitate meticulous hyperparameter tuning. Both of these challenges severely limit the applicability of such methods to complex, real-world domains. In this paper, we propose soft actor-critic, an off-policy actor-critic deep RL algorithm based on the maximum entropy reinforcement learning framework. In this framework, the actor aims to maximize expected reward while also maximizing entropy - that is, succeed at the task while acting as randomly as possible. Prior deep RL methods based on this framework have been formulated as Q-learning methods. By combining off-policy updates with a stable stochastic actor-critic formulation, our method achieves state-of-the-art performance on a range of continuous control benchmark tasks, outperforming prior on-policy and off-policy methods. Furthermore, we demonstrate that, in contrast to other off-policy algorithms, our approach is very stable, achieving very similar performance across different random seeds.


Compared to Pose2Seg #REFR , our method achieves better performance in both detection and segmentation tasks (Table 7).

6 in the main paper, we present several qualitative results on the OCHuman dataset #OTHEREFR .

In this section, we present quantitative results on the dataset.

Compared to a state-of-the-art segmentation method (Mask2Former #OTHEREFR ), our method shows a comparable accuracy in segmentation.

Compared to our baseline multitasking model (BaseNet-DPS), our approach still induces significant improvements in pose estimation and segmentation on this different dataset.

We show additional visual results of our methods on the COCO and OCHuman datasets.

Object-Centric Multi-Task Learning For Human Instances

Abstract: Human is one of the most essential classes in visual recognition tasks such as detection, segmentation, and pose estimation. Although much effort has been put into individual tasks, multi-task learning for these three tasks has been rarely studied. In this paper, we explore a compact multi-task network architecture that maximally shares the parameters of the multiple tasks via object-centric learning. To this end, we propose a novel query design to encode the human instance information effectively, called humancentric query (HCQ). HCQ enables for the query to learn explicit and structural information of human as well such as keypoints. Besides, we utilize HCQ in prediction heads of the target tasks directly and also interweave HCQ with the deformable attention in Transformer decoders to exploit a well-learned object-centric representation. Experimental results show that the proposed multi-task network achieves comparable accuracy to state-of-the-art task-specific models in human detection, segmentation, and pose estimation task, while it consumes less computational costs.

Pose2Seg: Detection Free Human Instance Segmentation

Abstract: The standard approach to image instance segmentation is to perform the object detection first, and then segment the object from the detection bounding-box. More recently, deep learning methods like Mask R-CNN perform them jointly. However, little research takes into account the uniqueness of the "human" category, which can be well defined by the pose skeleton. Moreover, the human pose skeleton can be used to better distinguish instances with heavy occlusion than using bounding-boxes. In this paper, we present a brand new pose-based instance segmentation framework for humans which separates instances based on human pose, rather than proposal region detection. We demonstrate that our pose-based framework can achieve better accuracy than the state-of-art detection-based approach on the human instance segmentation problem, and can moreover better handle occlusion. Furthermore, there are few public datasets containing many heavily occluded humans along with comprehensive annotations, which makes this a challenging problem seldom noticed by researchers. Therefore, in this paper we introduce a new benchmark "Occluded Human (OCHuman)", which focuses on occluded humans with comprehensive annotations including bounding-box, human pose and instance masks. This dataset contains 8110 detailed annotated human instances within 4731 images. With an average 0.67 MaxIoU for each person, OCHuman is the most complex and challenging dataset related to human instance segmentation. Through this dataset, we want to emphasize occlusion as a challenging problem for researchers to study.


Related Work 2.1. Ai-Synthesized Faces And Their Detection

Inspired by the superior performance of the StyleGAN series #REFR in synthesizing highresolution and high-quality images, StyleGAN architectures have been adopted for image editing, achieving highquality edited images.

Artificial Intelligence (AI)-synthesized faces can be either fully synthetic when the faces are generated from scratch using generative models, or locally manipulated, e.g.

when a single facial attribute or multiple attributes are modified by the model while the other attributes remain unchanged.

A wide variety of generative models, notably GANs #OTHEREFR and diffusion models #OTHEREFR , are nowadays able to generate high-resolution images from scratch with an unprecedented level of realism.

Among them, we mention InterFace-GAN #OTHEREFR and StyleCLIP [27] .

InterFaceGAN proposes a framework to interpret the disentangled face representation learned by the StyleGAN model and studies the properties of the facial semantics encoded in the latent space, showing that it is possible to edit the semantic attribute through linear subspace projection. StyleCLIP is a text-based interface for StyleGAN-based image manipulation.

StyleCLIP mainly uses the Contrastive Language-Image Pre-training (CLIP) model to edit the latent code through the user input language description, so as to achieve the purpose of editing the image.

With regard to the defences, several methods have been developed in the last years to discriminate between fake and real #OTHEREFR .

Attempts have also been made to attribute the GAN model and/or the type of architecture generating the image.

Classification Of Gan-Based Image Manipulations Via A Vit-Based Hybrid Architecture

Abstract: Classification of AI-manipulated content is receiving great attention, for distinguishing different types of manipulations. Most of the methods developed so far fail in the open-set scenario, that is when the algorithm used for the manipulation is not represented by the training set. In this paper, we focus on the classification of synthetic face generation and manipulation in open-set scenarios, and propose a method for classification with a rejection option. The proposed method combines the use of Vision Transformers (ViT) with a hybrid approach for simultaneous classification and localization. Feature map correlation is exploited by the ViT module, while a localization branch is employed as an attention mechanism to force the model to learn per-class discriminative features associated with the forgery when the manipulation is performed locally in the image. Rejection is performed by considering several strategies and analyzing the model output layers. The effectiveness of the proposed method is assessed for the task of classification of facial attribute editing and GAN attribution.

Analyzing And Improving The Image Quality Of Stylegan

Abstract: The style-based GAN architecture (StyleGAN) yields state-of-the-art results in data-driven unconditional generative image modeling. We expose and analyze several of its characteristic artifacts, and propose changes in both model architecture and training methods to address them. In particular, we redesign the generator normalization, revisit progressive growing, and regularize the generator to encourage good conditioning in the mapping from latent codes to images. In addition to improving image quality, this path length regularizer yields the additional benefit that the generator becomes significantly easier to invert. This makes it possible to reliably attribute a generated image to a particular network. We furthermore visualize how well the generator utilizes its output resolution, and identify a capacity problem, motivating us to train larger models for additional quality improvements. Overall, our improved model redefines the state of the art in unconditional image modeling, both in terms of existing distribution quality metrics as well as perceived image quality.

highquality edited images
StyleGAN architectures


To reduce uncertainty, which manifests as image blur #REFR , and produce sharper prediction image quality, recent models have estimated and sampled from these latent variables. Babaeizadeh et al.

To perform predictions over longer time horizons, Srivastava et al.

#OTHEREFR introduced autoencoders and LSTM units to model the temporal coherence.

Action-conditioned video prediction models provide further information to the prediction model [9; 23] for use in reinforcement learning and enabling model predictive control with video prediction systems #OTHEREFR . Villegas et al.

#OTHEREFR split frames into content and motion streams and devolved the problem into predicting the pose and dynamics of landmarks #OTHEREFR .

More recent methods have applied a stochastic assumption to the video prediction problem, stating that there are multiple possible outputs for a single given input, due to a set of latent variables.

#OTHEREFR applied this method to the optical flow method proposed in #OTHEREFR . Denton et al.

#OTHEREFR proposed a similar approach but with a simpler model using only basic layers, which was later built upon by #OTHEREFR for larger high-fidelity video prediction. Lee et al.

#OTHEREFR also built upon the method in #OTHEREFR by appending adversarial training techniques.

The existing video prediction models do not use the tactile sensation and assume the changes caused by PRI are fully observed in visual information and/or the models provide a prediction with high uncertainty.

b) Physical Robot Interaction: Opera et al.

Combining Vision And Tactile Sensation For Video Prediction

Abstract: In this paper, we explore the impact of adding tactile sensation to video prediction models for physical robot interactions. Predicting the impact of robotic actions on the environment is a fundamental challenge in robotics. Current methods leverage visual and robot action data to generate video predictions over a given time period, which can then be used to adjust robot actions. However, humans rely on both visual and tactile feedback to develop and maintain a mental model of their physical surroundings. In this paper, we investigate the impact of integrating tactile feedback into video prediction models for physical robot interactions. We propose three multimodal integration approaches and compare the performance of these tactile-enhanced video prediction models. Additionally, we introduce two new datasets of robot pushing that use a magneticbased tactile sensor for unsupervised learning. The first dataset contains visually identical objects with different physical properties, while the second dataset mimics existing robot-pushing datasets of household object clusters. Our results demonstrate that incorporating tactile feedback into video prediction models improves scene prediction accuracy and enhances the agent's perception of physical interactions and understanding of causeeffect relationships during physical robot interactions. Index Terms-Deep learning in robotics and automation, perception for grasping and manipulation, physical robot interaction, video prediction, force and tactile sensing.

Stochastic Variational Video Prediction

Abstract: Predicting the future in real-world settings, particularly from raw sensory observations such as images, is exceptionally challenging. Real-world events can be stochastic and unpredictable, and the high dimensionality and complexity of natural images requires the predictive model to build an intricate understanding of the natural world. Many existing methods tackle this problem by making simplifying assumptions about the environment. One common assumption is that the outcome is deterministic and there is only one plausible future. This can lead to low-quality predictions in real-world settings with stochastic dynamics. In this paper, we develop a stochastic variational video prediction (SV2P) method that predicts a different possible future for each sample of its latent variables. To the best of our knowledge, our model is the first to provide effective stochastic multi-frame prediction for real-world video. We demonstrate the capability of the proposed method in predicting detailed future frames of videos on multiple real-world datasets, both action-free and action-conditioned. We find that our proposed method produces substantially improved video predictions when compared to the same model without stochasticity, and to other stochastic video prediction methods. Our SV2P implementation will be open sourced upon publication.


XL-Net #REFR , which is also based on BERT, incorporates permutation operations to change the prediction order for each training iteration, allowing the model to learn more information across tokens.

Since the introduction of the Transformer architecture, it has become the dominant choice in natural language processing due to its parallelism and learning capabilities.

Generally, these transformer based pre-trained language models can be commonly classified into two types based on their training tasks: autoregressive language modeling and masked language modeling #OTHEREFR .

Given a sentence, which is composed of several tokens, the objective of masked language modeling, e.g., BERT #OTHEREFR and RoBERTa #OTHEREFR , refers to predicting the probability of a masked token given context information.

The most notable example of masked language modeling is BERT #OTHEREFR , which includes masked language modeling and next sentence prediction tasks.

RoBERTa #OTHEREFR , which uses the same architecture as BERT, improves its performance by increasing the amount of pre-training data and incorporating more challenging pre-training objectives.

While autoregressive language modeling, e.g., GPT-3 #OTHEREFR and OPT [45] , is to model the probability of the next token given previous tokens, hence, left-to-right language modeling.

Different from masked language models, autoregressive models are more suitable for generative tasks.

We will introduce more about autoregressive models in Section 4.1.1.

Decoder (GPT) ! " # Trm Trm Trm Trm Trm Trm ! " # … … … … ! " # Trm Trm Trm Trm Trm Trm ! " # … … … … Encoder (BERT) ! " # … Trm Trm Trm … Trm Trm Trm … Trm Trm Trm … ! " # … Encoder-Decoder (T5/BART)

A Comprehensive Survey Of Ai-Generated Content (Aigc): A History Of Generative Ai From Gan To Chatgpt

Abstract: Recently, ChatGPT, along with DALL-E-2 [1] and Codex [2],has been gaining significant attention from society. As a result, many individuals have become interested in related resources and are seeking to uncover the background and secrets behind its impressive performance. In fact, ChatGPT and other Generative AI (GAI) techniques belong to the category of Artificial Intelligence Generated Content (AIGC), which involves the creation of digital content, such as images, music, and natural language, through AI models. The goal of AIGC is to make the content creation process more efficient and accessible, allowing for the production of high-quality content at a faster pace. AIGC is achieved by extracting and understanding intent information from instructions provided by human, and generating the content according to its knowledge and the intent information. In recent years, large-scale models have become increasingly important in AIGC as they provide better intent extraction and thus, improved generation results. With the growth of data and the size of the models, the distribution that the model can learn becomes more comprehensive and closer to reality, leading to more realistic and high-quality content generation. This survey provides a comprehensive review on the history of generative models, and basic components, recent advances in AIGC from unimodal interaction and multimodal interaction. From the perspective of unimodality, we introduce the generation tasks and relative models of text and image. From the perspective of multimodality, we introduce the cross-application between the modalities mentioned above. Finally, we discuss the existing open problems and future challenges in AIGC.

Xlnet: Generalized Autoregressive Pretraining For Language Understanding

Abstract: With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language modeling. However, relying on corrupting the input with masks, BERT neglects dependency between the masked positions and suffers from a pretrain-finetune discrepancy. In light of these pros and cons, we propose XLNet, a generalized autoregressive pretraining method that (1) enables learning bidirectional contexts by maximizing the expected likelihood over all permutations of the factorization order and (2) overcomes the limitations of BERT thanks to its autoregressive formulation. Furthermore, XLNet integrates ideas from Transformer-XL, the state-of-the-art autoregressive model, into pretraining. Empirically, under comparable experiment setting, XLNet outperforms BERT on 20 tasks, often by a large margin, including question answering, natural language inference, sentiment analysis, and document ranking.


We use CentOS 7 for the operating system, Docker 19 for deployment, and TensorFlow 2.2 #REFR for training and embedding generation.

For training and testing, we partitioned the graph samples collected from 200M instructions with 70% for training and 30% for testing.

The entire training takes less than 20 hours on Nvidia Tesla V100.

After the model is trained, we deploy it to our prototype system to generate NPS embeddings for hundreds of billions of instructions. System Prototype.

Our prototype system runs on 10 Intel Xeon Platinum 8163 servers.

Each server has 48 CPU cores running on 2.5GHz with 800GB of memory.

Nps: A Framework For Accurate Program Sampling Using Graph Neural Network

Abstract: With the end of Moore's Law, there is a growing demand for rapid architectural innovations in modern processors, such as RISC-V custom extensions, to continue performance scaling. Program sampling is a crucial step in microprocessor design, as it selects representative simulation points for workload simulation. While SimPoint has been the de-facto approach for decades, its limited expressiveness with Basic Block Vector (BBV) requires time-consuming human tuning, often taking months, which impedes fast innovation and agile hardware development. This paper introduces Neural Program Sampling (NPS), a novel framework that learns execution embeddings using dynamic snapshots of a Graph Neural Network. NPS deploys AssemblyNet for embedding generation, leveraging an application's code structures and runtime states. AssemblyNet serves as NPS's graph model and neural architecture, capturing a program's behavior in aspects such as data computation, code path, and data flow. AssemblyNet is trained with a data prefetch task that predicts consecutive memory addresses. In the experiments, NPS outperforms SimPoint by up to 63%, reducing the average sampling error by 38%. Additionally, NPS demonstrates strong robustness with increased accuracy, reducing the expensive accuracy tuning overhead. Furthermore, NPS shows higher accuracy and generality than the state-of-the-art GNN approach in code behavior learning, enabling the generation of high-quality execution embeddings.

Tensorflow: A System For Large-Scale Machine Learning

Abstract: TensorFlow is a machine learning system that operates at large scale and in heterogeneous environments. Tensor-Flow uses dataflow graphs to represent computation, shared state, and the operations that mutate that state. It maps the nodes of a dataflow graph across many machines in a cluster, and within a machine across multiple computational devices, including multicore CPUs, general-purpose GPUs, and custom-designed ASICs known as Tensor Processing Units (TPUs). This architecture gives flexibility to the application developer: whereas in previous "parameter server" designs the management of shared state is built into the system, TensorFlow enables developers to experiment with novel optimizations and training algorithms. TensorFlow supports a variety of applications, with a focus on training and inference on deep neural networks. Several Google services use TensorFlow in production, we have released it as an open-source project, and it has become widely used for machine learning research. In this paper, we describe the TensorFlow dataflow model and demonstrate the compelling performance that TensorFlow achieves for several real-world applications.


The frame flow is computed on x s t−1 and x s t using image-based flow estimator #REFR .

We introduce the Context Adaptation and Propagation (CAP) module to make the model produce smooth videos. The illustration of CAP is shown in Fig. 5 .

First, the raw feature f r t−1 of the previous source frame is sequentially warped with the frame flow w t−1→t and the current local deformation w s→d local (Two-Step Warping).

Second, the warped previous raw featurẽ f r t−1 , hidden feature h t−1 , adapted feature f c t−1 and the current aligned feature f a t are concatenated.

We further feed the concatenated feature to the Context Adaptation module, which is composed of several convolution blocks, to get the feature in the same spatial and channel size with f a t .

Then, the feature is further refined with the Feature Refinement module.

After that, we get the adapted feature f c t for the current frame, and the hidden feature is updated with the adapted feature f c t using a ConvGRU block #OTHEREFR . The proposed CAP module can be formed as,

High-Fidelity And Freely Controllable Talking Head Video Generation

Abstract: PECHead (a). Frontalization Input Yaw Pitch Roll Input Neutral Smile Laugh Input Smile Expressionless Roll (b). Free pose and expression editing Figure 1. Presented herein are representative results showcasing the effectiveness of our proposed method in the tasks of frontalization, as well as free pose and expression editing. (a) FOMM [46] often produces face distortion issues, while our model PECHead generates high-fidelity results. (b) Our proposed framework empowers the generation of talking head videos that offer free control over the head pose and expression. More results can be found in the supplementary materials.

Raft: Recurrent All-Pairs Field Transforms For Optical Flow

Abstract: We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep network architecture for optical flow. RAFT extracts per-pixel features, builds multi-scale 4D correlation volumes for all pairs of pixels, and iteratively updates a flow field through a recurrent unit that performs lookups on the correlation volumes. RAFT achieves state-of-the-art performance, with strong cross-dataset generalization and high efficiency in inference time, training speed, and parameter count. Code is available \url{https://github.com/princeton-vl/RAFT}.


Evaluating Meta-Learning For The Principal

We now compare the learning behavior of a principal that is meta-trained (MAML) #REFR versus one trained with standard policy gradients (RL), in a simple Stackelberg game between a principal and an agent.

The Stackelberg game features a principal (leader) who acts first (intervene or not), and the agent (follower) acts second according to a best-response policy (Von Stackelberg, 2010).

Our goal is to learn an intervention policy that can adapt to different agent types and find the Stackelberg equilibrium.

The agent's actions are "cooperate" and "defect", while the principal can choose whether or not to intervene.

Assuming the row player is the agent and the column player is the principal, we define a 2 × 2 payoff matrix

No intervention (NI) Intervene (IN) Cooperate (C) u, 1 u + 1, 1 − c Defect (D) 1 − u, 0 −u, −c ,

Mermaide: Learning To Align Learners Using Model-Based Meta-Learning

Abstract: We study how a principal can efficiently and effectively intervene on the rewards of a previously unseen learning agent in order to induce desirable outcomes. This is relevant to many realworld settings like auctions or taxation, where the principal may not know the learning behavior nor the rewards of real people. Moreover, the principal should be few-shot adaptable and minimize the number of interventions, because interventions are often costly. We introduce MER-MAIDE, a model-based meta-learning framework to train a principal that can quickly adapt to out-ofdistribution agents with different learning strategies and reward functions. We validate this approach step-by-step. First, in a Stackelberg setting with a best-response agent, we show that meta-learning enables quick convergence to the theoretically known Stackelberg equilibrium at test time, although noisy observations severely increase the sample complexity. We then show that our model-based meta-learning approach is cost-effective in intervening on bandit agents with unseen explore-exploit strategies. Finally, we outperform baselines that use either meta-learning or agent behavior modeling, in both 0-shot and K = 1-shot settings with partial agent information.

Model-Agnostic Meta-Learning For Fast Adaptation Of Deep Networks

Abstract: We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning problems, including classification, regression, and reinforcement learning. The goal of meta-learning is to train a model on a variety of learning tasks, such that it can solve new learning tasks using only a small number of training samples. In our approach, the parameters of the model are explicitly trained such that a small number of gradient steps with a small amount of training data from a new task will produce good generalization performance on that task. In effect, our method trains the model to be easy to fine-tune. We demonstrate that this approach leads to state-of-the-art performance on two few-shot image classification benchmarks, produces good results on few-shot regression, and accelerates fine-tuning for policy gradient reinforcement learning with neural network policies.


We show the scores obtained through both the 3-class and 2-class modeling in Table III #REFR . We note that both approaches achieved similar scores.

The natural way to model the original 3-class setup is to use all three tags (from Table II ), which inherently gives per-class F1-scores.

But to compute the ANY-COMB scores, we just collapse the POS and COMB labels to the same ANY-COMB label and compute the ANY-COMB F1-score.

A slightly different way is to use a 2-class set up during the training and testing through a different labeling scheme upfront.

For the POS classifier, we simply use two tags: @POS@ and @NON-POS@, where the latter is the tag given to instances in the union of COMB and NOCOMB.

For the ANY-COMB classifier, we use a new tag @ANY-COMB@ for instances in the union of POS and COMB classes and the @NOCOMB@ is the tag for the other class.

The the 3-way setup results in a slightly better POS score and the 2-way labeling improves by 1.6% in ANY-COMB F1-score compared with the 3-class approach.

Precision and recall are also close to each other but overall recall is slightly less than precision.

Next, we compare our results with prior state-of-the-art in Table IV , where the last row contains the F1-scores of our 3-way model (from Table III ).

We see that for both POS and ANY-COMB settings, our Seq2Rel model has better performance.

Compared with the penultimate row, which shows the prior SoTA score on this dataset, we have an almost 5% improvement in POS score even though our model is end-to-end while all other rows in the table assume drugs names are spotted.

End-To-End N-Ary Relation Extraction For Combination Drug Therapies

Abstract: Combination drug therapies are treatment regimens that involve two or more drugs, administered more commonly for patients with cancer, HIV, malaria, or tuberculosis. Currently there are over 350K articles in PubMed that use the combination drug therapy MeSH heading with at least 10K articles published per year over the past two decades. Extracting combination therapies from scientific literature inherently constitutes an n-ary relation extraction problem. Unlike in the general n-ary setting where n is fixed (e.g., drug-gene-mutation relations where n = 3), extracting combination therapies is a special setting where n ≥ 2 is dynamic, depending on each instance. Recently, Tiktinsky et al. (NAACL 2022) introduced a first of its kind dataset, CombDrugExt, for extracting such therapies from literature. Here, we use a sequence-to-sequence style end-to-end extraction method to achieve an F1-Score of 66.7% on the CombDrugExt test set for positive (or effective) combinations. This is an absolute ≈ 5% F1score improvement even over the prior best relation classification score with spotted drug entities (hence, not end-to-end). Thus our effort introduces a state-of-the-art first model for end-to-end extraction that is already superior to the best prior non end-toend model for this task. Our model seamlessly extracts all drug entities and relations in a single pass and is highly suitable for dynamic n-ary extraction scenarios.

Bert: Pre-Training Of Deep Bidirectional Transformers For Language Understanding

Abstract: We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications. ::: BERT is conceptually simple and empirically powerful. It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5% (7.7% point absolute improvement), MultiNLI accuracy to 86.7% (4.6% absolute improvement), SQuAD v1.1 question answering Test F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1 (5.1 point absolute improvement).


In this setting, kernel operators have demonstrated an impressive ability in capturing complex nonlinear phenomena #REFR .

Kernel methods have a long-standing history in machine learning #OTHEREFR , with established applications in detecting nonlinear data manifolds and order reduction [Coifman and Lafon, 2006a] .

Recently, researchers have studied dynamics-dependent kernels in the context of dynamical systems #OTHEREFR , with a particular focus on delay coordinate space.

, organized in a time-ordered manner, where each F (x i ) ∈ R d .

The value of x n is determined by the function Φ n∆t (x 0 ), where ∆t is the interval at which the data is sampled.

A delay coordinate map is constructed from F by embedding X in a manifold in R Qd as

Characterizing The Load Profile In Power Grids By Koopman Mode Decomposition Of Interconnected Dynamics

Abstract: Electricity load forecasting is crucial for effectively managing and optimizing power grids. Over the past few decades, various statistical and deep learning approaches have been used to develop load forecasting models. This paper presents an interpretable machine learning approach that identifies load dynamics using data-driven methods within an operator-theoretic framework. We represent the load data using the Koopman operator, which is inherent to the underlying dynamics. By computing the corresponding eigenfunctions, we decompose the load dynamics into coherent spatiotemporal patterns that are the most robust features of the dynamics. Each pattern evolves independently according to its single frequency, making its predictability based on linear dynamics. We emphasize that the load dynamics are constructed based on coherent spatiotemporal patterns that are intrinsic to the dynamics and are capable of encoding rich dynamical features at multiple time scales. These features are related to complex interactions over interconnected power grids and different exogenous effects. To implement the Koopman operator approach more efficiently, we cluster the load data using a modern kernel-based clustering approach and identify power stations with similar load patterns, particularly those with synchronized dynamics. We evaluate our approach using a large-scale dataset from a renewable electric power system within the continental European electricity system and show that the Koopman-based approach outperforms a deep learning (LSTM) architecture in terms of accuracy and computational efficiency.

Dynamics-Adapted Cone Kernels

Abstract: We present a family of kernels for analysis of data generated by dynamical systems. These so-called cone kernels feature a dependence on the dynamical vector field operating in the phase space manifold, estimated empirically through finite differences of time-ordered data samples. In particular, cone kernels assign strong affinity to pairs of samples whose relative displacement vector lies within a narrow cone aligned with the dynamical vector field. The outcome of this explicit dependence on the dynamics is that, in a suitable asymptotic limit, Laplace--Beltrami operators for data analysis constructed from cone kernels generate diffusions along the integral curves of the dynamical vector field. This property is independent of the observation modality, and endows these operators with invariance under a weakly restrictive class of transformations of the data (including conformal transformations), while it also enhances their capability to extract intrinsic dynamical timescales via eigenfunctions. Here, we ...


In our simulations, the one-bit channel estimation method developed in #REFR is adopted to provide the estimated CSI.

To evaluate the data detection performance of the proposed methods in the multiuser massive MIMO system, we compare the following detection methods: 1) Naive learning-based one-bit ML 2) ADL-based one-bit ML (proposed) 3) ADL-based one-bit ML with estimated SNR (proposed) 4) Minimum-Center-Distance (MCD) #OTHEREFR 5) One-bit ZF with perfect CSI #OTHEREFR 6) One-bit ML with perfect CSI (optimal one-bit detection) 7) One-bit ML with estimated CSI 8) Infinite-bit ML with perfect CSI (optimal detection) We note that the learning-based methods: 1) Naive onebit ML, 2) ADL one-bit ML, 3) ADL one-bit ML with estimated SNR, and 4) MCD, do not require explicit channel estimation; however, the other methods either assume perfect CSI or estimated CSI at the AP.

The learning-based methods transmit N tr pilot signals per each training symbol vector, which requires KN tr pilot signals in total.

Accordingly, we consider that the conventional one-bit ML detection with an estimated channel also uses KN tr pilot signals to estimate the channel.

For readability of the curves, we compare MCD for the 16-QAM case shown in Fig. 8 . Fig.

6 presents the SER results for N r = 32 antennas, N u = 4 users, N tr = 45 pilot signals, and 4-QAM. As expected from Fig.

5 , the naive-learning approach shows the catastrophic result from the medium to high SNR due to the large number of zero-valued likelihood functions.

The onebit ZF detection that applies the pseudo-inverse matrix of the perfectly-known channel matrix onto the one-bit observations shows the large performance degradation with the error floor at the medium and high SNR regime.

The one-bit ML detection with the one-bit estimated channels shows a larger deviation from the optimal one-bit ML detection with perfect CSI as the SNR increases due to the channel estimation error.

Adaptive Learning-Based Maximum Likelihood And Channel-Coded Detection For Massive Mimo Systems With One-Bit Adcs

Abstract: In this paper, we propose a learning-based detection framework for uplink massive multiple-input and multiple-output (MIMO) systems with one-bit analog-to-digital converters. The learning-based detection only requires counting the occurrences of the quantized outputs of-1 and +1 for estimating a likelihood probability at each antenna. Accordingly, the key advantage of this approach is to perform maximum likelihood detection without explicit channel estimation which has been one of the primary challenges of one-bit quantized systems. The learning in the high signal-to-noise ratio (SNR) regime, however, needs excessive training to estimate the extremely small likelihood probabilities. To address this drawback, we propose a dither-andlearning technique to estimate likelihood functions from dithered signals. First, we add a dithering signal to artificially decrease the SNR and then infer the likelihood function from the quantized dithered signals by using an SNR estimate derived from a deep neural network-based offline estimator. We extend our technique by developing an adaptive dither-and-learning method that updates the dithering power according the patterns observed in the quantized dithered signals. The proposed framework is also applied to state-of-the-art channel-coded MIMO systems by computing a bit-wise and user-wise log-likelihood ratio from the refined likelihood probabilities. Simulation results validate the detection performance of the proposed methods in both uncoded and coded systems. Index Terms-Massive MIMO, one-bit ADC, dithering, maximum likelihood detection, deep neural network. I. INTRODUCTION Massive MIMO systems for sub-6 GHz wireless communications [2], [3] and millimeter wave (mmWave) communications [4]-[6] have been considered as one of the emerging technologies for future communications because of the outstanding gain in spectral efficiency and capacity [7]. As wireless communication systems continue to grow in popularity and importance, there is a need to investigate communication systems that are not only reliable and high-performing, but also energy-efficient for various future wireless applications such as internet-of-things (IoT), extended reality, and smart grid [8], [9]. Because of the small wavelength of mmWave

Near Maximum-Likelihood Detector And Channel Estimator For Uplink Multiuser Massive Mimo Systems With One-Bit Adcs

Abstract: In massive multiple-input multiple-output (MIMO) systems, it may not be power efficient to have a pair of high-resolution analog-to-digital converters (ADCs) for each antenna element. In this paper, a near maximum likelihood (nML) detector for uplink multiuser massive MIMO systems is proposed where each antenna is connected to a pair of one-bit ADCs, i.e., one for each real and imaginary component of the baseband signal. The exhaustive search over all the possible transmitted vectors required in the original maximum likelihood (ML) detection problem is relaxed to formulate an ML estimation problem. Then, the ML estimation problem is converted into a convex optimization problem which can be efficiently solved. Using the solution, the base station can perform simple symbol-by-symbol detection for the transmitted signals from multiple users. To further improve detection performance, we also develop a two-stage nML detector that exploits the structures of both the original ML and the proposed (one-stage) nML detectors. Numerical results show that the proposed nML detectors are efficient enough to simultaneously support multiple uplink users adopting higher-order constellations, e.g., 16 quadrature amplitude modulation. Since our detectors exploit the channel state information as part of the detection, an ML channel estimation technique with one-bit ADCs that shares the same structure with our proposed nML detector is also developed. The proposed detectors and channel estimator provide a complete low power solution for the uplink of a massive MIMO system.


The training set D is derived from 2 publicly available, largescale (and biased) image datasets, CelebA and LFWA #REFR .

In this scenario, the model's task is to predict the sensitive attribute, i.e.

A = Y , and the reference dataset T is fair.

Specifically, we perform a gender classification task from facial images by means of a deep convolutional network using FairShap for data re-weighting.

We use Inception Resnet V1 #OTHEREFR as the gender classification model.

FairFace #OTHEREFR is the reference (and fair) dataset T .

Gender is both the protected attribute (A) and the target variable (Y ). The pipeline (see Fig.

2 for an illustration) to obtain the instance-level pre-processing weights in this scenario is as follows:

(1) we pre-train an Inception-v1 model with the CelebA dataset;

(2) we use this model to obtain the embeddings for the LFWA and FairFace datasets;

Fairshap: A Data Re-Weighting Approach For Algorithmic Fairness Based On Shapley Values

Abstract: In this paper, we propose FairShap, a novel and interpretable pre-processing (re-weighting) method for fair algorithmic decision-making through data valuation. FairShap is based on the Shapley Value, a well-known mathematical framework from game theory to achieve a fair allocation of resources. Our approach is easily interpretable, as it measures the contribution of each training data point to a predefined fairness metric. We empirically validate FairShap on several state-of-the-art datasets of different nature, with different training scenarios and models. The proposed approach outperforms other methods, yielding significantly fairer models with similar levels of accuracy. In addition, we illustrate FairShap's interpretability by means of histograms and latent space visualizations. We believe this work represents a promising direction in interpretable, model-agnostic approaches to algorithmic fairness.

Deep Learning Face Attributes In The Wild

Abstract: Predicting face attributes in the wild is challenging due to complex face variations. We propose a novel deep learning framework for attribute prediction in the wild. It cascades two CNNs, LNet and ANet, which are fine-tuned jointly with attribute tags, but pre-trained differently. LNet is pre-trained by massive general object categories for face localization, while ANet is pre-trained by massive face identities for attribute prediction. This framework not only outperforms the state-of-the-art with a large margin, but also reveals valuable facts on learning face representation. (1) It shows how the performances of face localization (LNet) and attribute prediction (ANet) can be improved by different pre-training strategies. (2) It reveals that although the filters of LNet are fine-tuned only with image-level attribute tags, their response maps over entire images have strong indication of face locations. This fact enables training LNet for face localization with only image-level annotations, but without face bounding boxes or landmarks, which are required by all attribute recognition works. (3) It also demonstrates that the high-level hidden neurons of ANet automatically discover semantic concepts after pre-training with massive face identities, and such concepts are significantly enriched after fine-tuning with attribute tags. Each attribute can be well explained with a sparse linear combination of these concepts.


More specifically, in #REFR , several relaying schemes were developed to address the blockage issue in THz communications, in which the performance gain from using relaying was rigorously evaluated.

In #OTHEREFR , to capture the obstruction of walls and human bodies, etc., the coverage probability was studied by using the stochastic geometry.

The analytical results revealed the detrimental impact of the node density, transmission dis-tance, and atmosphere upon the coverage probability.

Recently, some prior works have tried to solve this dilemma.

In #OTHEREFR , massive multiple-input multiple-output (MIMO) aided THz communications were proposed for the blockage mitigation and a network sum-rate based optimization scheme was devised by considering only statistical channel state information (CSI).

Moreover, in order to substantially overcome blockage impairment, one basic idea is altering the communication links so as to circumvent harsh propagation environment, which can be realized by combining relaying, reconfigurable intelligent surfaces (RIS), unmanned aerial vehicles (UAVs), etc. #OTHEREFR .

In #OTHEREFR , RIS was employed to aid THz communications and it is uncovered that the THz communications performance can be significantly enhanced by increasing the number of RIS's elements.

Besides, a block coordinate searching algorithm was developed to jointly optimize the coordinates and phase shifts of RIS for THz communications in #OTHEREFR .

Some other latest works #OTHEREFR , #OTHEREFR also verified that integrating RIS into THz systems not only avoids the effect of blockage but also expands the coverage area.

Moreover, in #OTHEREFR , the UAV was deployed to assist THz communications and the total transmission delay between the UAV and the ground user was minimized through optimizing the location, bandwidth and power, which can be solved with alternating algorithm.

Unfortunately, most of prior optimization methodologies for designing THz communication systems have a prohibitively high computational complexity because of the complex fading characteristics of THz communications.

Performance Analysis And Optimal Design Of Harq-Ir-Aided Terahertz Communications

Abstract: Terahertz (THz) communications are envisioned to be a promising technology for 6G thanks to its broad bandwidth. However, the large path loss, antenna misalignment, and atmospheric influence of THz communications severely deteriorate its reliability. To address this, hybrid automatic repeat request (HARQ) is recognized as an effective technique to ensure reliable THz communications. This paper delves into the performance analysis of HARQ with incremental redundancy (HARQ-IR)-aided THz communications in the presence/absence of blockage. More specifically, the analytical expression of the outage probability of HARQ-IR-aided THz communications is derived, with which the asymptotic outage analysis is enabled to gain meaningful insights, including diversity order, power allocation gain, modulation and coding gain, etc. Then the long term average throughput (LTAT) is expressed in terms of the outage probability based on renewal theory. Moreover, to combat the blockage effects, a multi-hop HARQ-IR-aided THz communication scheme is proposed and its performance is examined. To demonstrate the superiority of the proposed scheme, the other two HARQ-aided schemes, i.e., Type-I HARQ and HARQ with chase combining (HARQ-CC), are used for benchmarking in the simulations. In addition, a deep neural network (DNN) based outage evaluation framework with low computational complexity is devised to reap the benefits of using both asymptotic and simulation results in low and high outage regimes, respectively. This novel outage evaluation framework is finally employed for the optimal rate selection, which outperforms the asymptotic based optimization. Index Terms-Blockage, hybrid automatic repeat request (HARQ), outage probability, terahertz (THz) communications. I. INTRODUCTION T O meet the unprecedented increase of wireless devices and data traffic, there is an urgent need to boost the transmission rate. To accommodate the ever-growing demand, terahertz (THz) communications have emerged as a potential technique to support the peak data rate of more than 1 Tbps [1]-[3]. Different from the conventional low-frequency channel propagation characteristics, THz communications are susceptible to atmospheric turbulence, pointing errors, rain attenuation, etc. [4]-[12]. As uncovered by [13], [14], these

Relay-Based Blockage And Antenna Misalignment Mitigation In Thz Wireless Communications

Abstract: The proliferation of wireless devices in recent years has caused a spectrum shortage, which led the scientific community to explore the potential of using terahertz (THz) communications. However, THz systems suffer from severe path attenuation, blockage, and antenna misalignment. In this paper, we present a relay-based blockage and antenna misalignment mitigation approach. In more detail, two relay selection policies are employed, namely best and random relay selection. The system performance under both policies is evaluated and compared in terms of average throughput and the probability that throughput of a link is below the quality of service (QoS) threshold, using Monte Carlo simulations. It was observed that the effect of both blockage and misalignment can be mitigated using relays. Moreover, the gain of using relaying to mitigate blockage is much more significant.


One solution is to design a provably correct compiler, but this is a complex and long-term endeavour #REFR .

However, the original specification in the Yellow Paper #OTHEREFR has some known shortcomings: (i) it is hard to read and does not provide a formal semantics of the EVM and the bytecode; (ii) the lack of a formal semantics makes it hard for Ethereum client developers to guarantee that they interpret the Yellow Paper in a consistent way; (iii) designing compilers from high-level languages (e.g., Solidity 1 ) to EVM bytecode without a formal semantics is error-prone and, without a precise semantics of the EVM, it is hard to design certified compilers (preserving of semantics from a high to a low-level language.).

One can argue that existing implementations of the EVM (e.g., in Go, Java) provide a de facto semantics for it.

Whilst this is true to some extent, such implementations do not enable formal reasoning about bytecode.

Furthermore, whilst smart contracts can be written in highlevel languages like Solidity, they must be compiled into EVM bytecode before being executed on the EVM.

Tools for checking safety properties (e.g., absence of overflow, division by zero, etc) at the Solidity level are problematic if they cannot guarantee such properties hold at the bytecode level.

Alternatively we can provide techniques, supported by tools, to reason about properties of the bytecode.

This is what we propose to do in this work.

We present a complete and formal specification of the EVM in Dafny, available at https://github.com/ConsenSys/evm-dafny.

We provide a formal semantics where the meaning of an instruction is given as a partial function that maps states to states.

Formal And Executable Semantics Of The Ethereum Virtual Machine In Dafny

Abstract: The Ethereum protocol implements a replicated state machine. The network participants keep track of the system state by: 1) agreeing on the sequence of transactions to be processed and 2) computing the state transitions that correspond to the sequence of transactions. Ethereum transactions are programs, called smart contracts, and computing a state transition requires executing some code. The Ethereum Virtual Machine (EVM) provides this capability and can execute programs written in EVM bytecode. We present a formal and executable semantics of the EVM written in the verification-friendly language Dafny: it provides (i) a readable, formal and verified specification of the semantics of the EVM; (ii) a framework to formally reason about bytecode.

A Formally Verified Compiler Back-End

Abstract: This article describes the development and formal verification (proof of semantic preservation) of a compiler back-end from Cminor (a simple imperative intermediate language) to PowerPC assembly code, using the Coq proof assistant both for programming the compiler and for proving its soundness. Such a verified compiler is useful in the context of formal methods applied to the certification of critical software: the verification of the compiler guarantees that the safety properties proved on the source code hold for the executable compiled code as well.


Theorem 10. The Capacity Of The Trapdoor Channel With Delayed Feedback Of Two Time Instances Is

As explained in Remark 2, the scheme is based on the posterior matching principle, and the exact details regarding the constriction of the coding scheme are given in #REFR .

In other words, the delayed feedback capacity in Theorem 10 does not seem to be the feedforward capacity, which remains an open problem.

The achievability proof of Theorem 10 is based on the Q-graph lower bound, which was presented in Section IV-B.

That is, the lower bound was established by showing that a particular graphbased encoder, given by the Q-graph in #OTHEREFR and the input distribution in #OTHEREFR , provides an achievable rate of log 2 (3/2).

This graph-based encoder implies a simple coding scheme that, in our case, achieves the capacity.

Following the formulation in Section IV-A, we present in Fig.

5 the trapdoor channel with delayed feedback of two time instances as a new unifilar FSC with instantaneous feedback.

For the new FSC, it is interesting to note that the capacity of each individual channel (per state) is zero.

Nevertheless, as we already demonstrated, the capacity of the overall FSC is not zero.

Capacity Of Finite-State Channels With Delayed Feedback

Abstract: In this paper, we investigate the capacity of finite-state channels (FSCs) in presence of delayed feedback. We show that the capacity of a FSC with delayed feedback can be computed as that of a new FSC with instantaneous feedback and an extended state. Consequently, graph-based methods to obtain computable upper and lower bounds on the delayed feedback capacity of unifilar FSCs are proposed. Based on these methods, we establish that the capacity of the trapdoor channel with delayed feedback of two time instances is given by log 2 3 2. In addition, we derive an analytical upper bound on the delayed feedback capacity of the binary symmetric channel with a no consecutive ones input constraint. This bound also serves as a novel upper bound on its non-feedback capacity, which outperforms all previously known bounds. Lastly, we demonstrate that feedback does improve the capacity of the dicode erasure channel. This paper was presented in part at the 2022 IEEE International Symposium on Information Theory [1]. 1 The Q-graph is an auxiliary directed graph that is used to map channel output sequences onto one of the graph nodes (e.g. see Fig. 4).

Graph-Based Encoders And Their Performance For Finite-State Channels With Feedback

Abstract: The capacity of unifilar finite-state channels in the presence of feedback is investigated. We derive a new evaluation method to extract graph-based encoders with their achievable rates, and to compute upper bounds to examine their performance. The evaluation method is built upon a recent methodology to derive simple bounds on the capacity using auxiliary directed graphs. While it is not clear whether the upper bound is convex, we manage to formulate it as a convex optimization problem using transformation of the argument with proper constraints. The lower bound is formulated as a non-convex optimization problem, yet, any feasible point to the optimization problem induces a graph-based encoder. In all examples, the numerical results show near-tight upper and lower bounds that can be easily converted to analytic results. For the non-symmetric Trapdoor channel and binary fading channels (BFCs), new capacity results are established by computing the corresponding bounds. For all other instances, including the Ising channel, the near-tightness of the achievable rates is shown via a comparison with corresponding upper bounds. Finally, we show that any graph-based encoder implies a simple coding scheme that is based on the posterior matching principle and achieves the lower bound.


We observe that all feature extractors considered can somewhat succeed at the task, including the ResNet-50 with random weights, which we expect to still produce valuable features #REFR .

CelebA sensitivities Tab. 2 presents results as averaged over all attributes (i.e.

with a = hat and b being all other attributes, then a = glasses and b all others, etc.).

The results clearly show that neuron granularity is crucial for success as fd, which operates layer-wise, falls short against dna-emd and dna-fd.

Averaged over all attributes, our approach can discard 95.5% of the distance over the attributes on which we remove sensitivity, while only causing a 9.6% of deviation in distances over other attributes.

dna-emd performs slightly better than dna-fd, but both do very well.

However, we obtain the best results using self-supervised models which are likely to produce more informative features.

Finding similar images Qualitatively, we expect neurons to react to general and consistent features, which should also apply to comparing image pairs and different datasets.

To verify this, we compare image pairs from a different dataset, FFHQ #OTHEREFR , with and without specific attributes (eyeglasses and wearing hat) and select the neuron(s) with the highest dna-emd sensitivity on CelebA. Using the selected neurons, we compare the DNA hist. of a selected reference image to DNAs hist. of 2000 random FFHQ samples. We present our results in Fig. 6 .

We can verify that very few neurons are required to focus on high-level semantic attributes, even when selected on a different dataset.

We still observe some errors, possibly due to neurons reacting to several attributes simultaneously.

Visual Dna: Representing And Comparing Images Using Distributions Of Neuron Activations

Abstract: Selecting appropriate datasets is critical in modern computer vision. However, no general-purpose tools exist to evaluate the extent to which two datasets differ. For this, we propose representing images-and by extension datasetsusing Distributions of Neuron Activations (DNAs). DNAs fit distributions, such as histograms or Gaussians, to activations of neurons in a pre-trained feature extractor through which we pass the image(s) to represent. This extractor is frozen for all datasets, and we rely on its generally expressive power in feature space. By comparing two DNAs, we can evaluate the extent to which two datasets differ with granular control over the comparison attributes of interest, providing the ability to customise the way distances are measured to suit the requirements of the task at hand. Furthermore, DNAs are compact, representing datasets of any size with less than 15 megabytes. We demonstrate the value of DNAs by evaluating their applicability on several tasks, including conditional dataset comparison, synthetic image evaluation, and transfer learning, and across diverse datasets, ranging from synthetic cat images to celebrity faces and urban driving scenes. Real horses Fake horse Person 1 Person 2 (b) The DNA can also be used to compare individual images to datasets-for example, to measure the realism and semantic consistency of a synthetic image-or pairs of images-for example, to verify the presence of similar attributes such as smiling or wearing a hat.

What'S Hidden In A Randomly Weighted Neural Network?

Abstract: Training a neural network is synonymous with learning the values of the weights. By contrast, we demonstrate that randomly weighted neural networks contain subnetworks which achieve impressive performance without ever training the weight values. Hidden in a randomly weighted Wide ResNet-50 we show that there is a subnetwork (with random weights) that is smaller than, but matches the performance of a ResNet-34 trained on ImageNet. Not only do these"untrained subnetworks"exist, but we provide an algorithm to effectively find them. We empirically show that as randomly weighted neural networks with fixed weights grow wider and deeper, an"untrained subnetwork"approaches a network with learned weights in accuracy. Our code and pretrained models are available at https://github.com/allenai/hidden-networks.


Ec.6. Additional Experiments On Parameter Estimation

In our problem, the inventory is limited, the numbers of arriving customers (M t ) t∈[T ] are general, and the assortment set can dynamically change within each epoch whenever stochastic stockout events occur. To apply the estimator in #REFR to our setup, we consider three natural extensions.

In this section, we compare our attraction parameter estimatorv = (v 1, , . . .

,v N, ) proposed in Sec- the same assortment until a no-purchase option occurs, and then record the number of purchases of each assorted product before the first no-purchase during this epoch.

Then the average of each product's summary statistics recorded in all previous epochs where that product is assorted gives an estimator for the corresponding attraction parameter. #OTHEREFR

(2019) shows that this estimator is consistent in their setup where there is no inventory limit and the assortment set is fixed in each epoch.

These extensions are all based on a similar epoch structure: in each epoch , we offer the same inventory level at the beginning of every cycle until a stopping criterion is met and then we record a summary statistic v i, for each assorted product i ∈ [N ] .

With slight abuse of notations, we still denote the subset of epochs up to in which product i is assorted by T i, and its cardinality by T i, .

The estimators in the three extensions all have the same formv i, := 1/T i,

just with slightly different definitions of epoch stopping criteria and summary statistics v i,τ 's.

Online Joint Assortment-Inventory Optimization Under Mnl Choices

Abstract: We study an online joint assortment-inventory optimization problem, in which we assume that the choice behavior of each customer follows the Multinomial Logit (MNL) choice model, and the attraction parameters are unknown a priori. The retailer makes periodic assortment and inventory decisions to dynamically learn from the realized demands about the attraction parameters while maximizing the expected total profit over time. In this paper, we propose a novel algorithm that can effectively balance the exploration and exploitation in the online decision-making of assortment and inventory. Our algorithm builds on a new estimator for the MNL attraction parameters, a novel approach to incentivize exploration by adaptively tuning certain known and unknown parameters, and an optimization oracle to static single-cycle assortment-inventory planning problems with given parameters. We establish a regret upper bound for our algorithm and a lower bound for the online joint assortment-inventory optimization problem, suggesting that our algorithm achieves nearly optimal regret rate, provided that the static optimization oracle is exact. Then we incorporate more practical approximate static optimization oracles into our algorithm, and bound from above the impact of static optimization errors on the regret of our algorithm. At last, we perform numerical studies to demonstrate the effectiveness of our proposed algorithm.

Mnl-Bandit: A Dynamic Learning Approach To Assortment Selection

Abstract: We consider a dynamic assortment selection problem where in every round the retailer offers a subset (assortment) of N substitutable products to a consumer, who selects one of these products accord...


Although we demonstrate the feasibility of using QBO to optimize hyperparameters of a machine learning model, the potential benefits of employing quantum kernels over classical machine learning methods in tasks using classical data remain uncertain #REFR .

In this study, we apply QGP models to one and multi-dimensional regression problems and show that they can be used as a surrogate model for BO to create a QBO.

We demonstrate that QBO can be used to solve real-world hyperparameter optimization problems.

Our encoding strategy allows for effectively using the variational parameters of the data embedding circuit as hyperparameters for the quantum kernels.

In our simulations, we observe that the posterior variance of the QGP remains intact under the influence of samplingnoise and even for the calculation NISQ devices, although the influence of the various error sources in the latter affect the result.

Nevertheless, since the results from the sampling-based simulations can be seen as an upper-bound for future hardware capabilities, the outlook is optimistic.

However, it is reasonable to expect that QBO may provide advantages in problems where quantum data can be leveraged to achieve a quantum advantage #OTHEREFR .

Notably, QBO is potentially well-suited for active learning tasks in expensive molecular simulations, where the evaluation of the potential energy surface is based on quantum mechanics and is computationally expensive #OTHEREFR .

The performance of the QGP model remains unexplored in several avenues within this work.

For example, the choice of feature map is a crucial aspect and it has been shown that choosing problem-specific feature map with an inductive bias that is tailored to the dataset has various advantages such as improved performance and trainability #OTHEREFR .

It is also known that using parametrized feature maps require special care when scaling the number of qubits which can lead to exponential concentration #OTHEREFR .

Quantum Gaussian Process Regression For Bayesian Optimization

Abstract: Gaussian process regression is a well-established Bayesian machine learning method. We propose a new approach to Gaussian process regression using quantum kernels based on parameterized quantum circuits. By employing a hardware-efficient feature map and careful regularization of the Gram matrix, we demonstrate that the variance information of the resulting quantum Gaussian process can be preserved. We also show that quantum Gaussian processes can be used as a surrogate model for Bayesian optimization, a task that critically relies on the variance of the surrogate model. To demonstrate the performance of this quantum Bayesian optimization algorithm, we apply it to the hyperparameter optimization of a machine learning model which performs regression on a real-world dataset. We benchmark the quantum Bayesian optimization against its classical counterpart and show that quantum version can match its performance.

Sampling-Based Sublinear Low-Rank Matrix Arithmetic Framework For Dequantizing Quantum Machine Learning

Abstract: We present an algorithmic framework generalizing quantum-inspired polylogarithmic-time algorithms on low-rank matrices. Our work follows the line of research started by Tang's breakthrough classical algorithm for recommendation systems [STOC'19]. The main result of this work is an algorithm for singular value transformation on low-rank inputs in the quantum-inspired regime, where singular value transformation is a framework proposed by Gilyen et al. [STOC'19] to study various quantum speedups. Since singular value transformation encompasses a vast range of matrix arithmetic, this result, combined with simple sampling lemmas from previous work, suffices to generalize all results dequantizing quantum machine learning algorithms to the authors' knowledge. Via simple black-box applications of our singular value transformation framework, we recover the dequantization results on recommendation systems, principal component analysis, supervised clustering, low-rank matrix inversion, low-rank semidefinite programming, and support vector machines. We also give additional dequantizations results on low-rank Hamiltonian simulation and discriminant analysis.


In Figure 2a , we show the discounted return achieved by POMCP (with and without rule bias) when the number of particles is varied between 2 8 and 2 #REFR .

Introducing learned ASP axioms does not significantly affect the performance of the solver.

This is probably due to the limited number of actions involved in the task (only 3, i.e., advance, check and recharge), thus few branches must be explored at each simulation step.

Figure 2b shows the discounted return achieved by POMCP when the length of the path increases, while the number of particles is constantly 2 #OTHEREFR as in the training setup.

In this case, the solver performs significantly better when learned rules are employed (the discounted return increases up to 20% when the path length is 75, i.e., almost double than the training scenario).

In fact, as the path length increases, the risk of battery discharge is higher, since failing to recharge is more probable.

Learning Logic Specifications For Soft Policy Guidance In Pomcp

Abstract: Partially Observable Monte Carlo Planning (POMCP) is an efficient solver for Partially Observable Markov Decision Processes (POMDPs). It allows scaling to large state spaces by computing an approximation of the optimal policy locally and online, using a Monte Carlo Tree Search based strategy. However, POMCP suffers from sparse reward function, namely, rewards achieved only when the final goal is reached, particularly in environments with large state spaces and long horizons. Recently, logic specifications have been integrated into POMCP to guide exploration and to satisfy safety requirements. However, such policy-related rules require manual definition by domain experts, especially in real-world scenarios. In this paper, we use inductive logic programming to learn logic specifications from traces of POMCP executions, i.e., sets of belief-action pairs generated by the planner. Specifically, we learn rules expressed in the paradigm of answer set programming. We then integrate them inside POMCP to provide soft policy bias toward promising actions. In the context of two benchmark scenarios, rocksample and battery, we show that the integration of learned rules from small task instances can improve performance with fewer Monte Carlo simulations and in larger task instances.

Iterative Learning Of Answer Set Programs From Context Dependent Examples

Abstract: In recent years, several frameworks and systems have been proposed that extend Inductive Logic Programming (ILP) to the Answer Set Programming (ASP) paradigm. In ILP, examples must all be explained by a hypothesis together with a given background knowledge. In existing systems, the background knowledge is the same for all examples; however, examples may be context-dependent. This means that some examples should be explained in the context of some information, whereas others should be explained in different contexts. In this paper, we capture this notion and present a context-dependent extension of the Learning from Ordered Answer Sets framework. In this extension, contexts can be used to further structure the background knowledge. We then propose a new iterative algorithm, ILASP2i, which exploits this feature to scale up the existing ILASP2 system to learning tasks with large numbers of examples. We demonstrate the gain in scalability by applying both algorithms to various learning tasks. Our results show that, compared to ILASP2, the newly proposed ILASP2i system can be two orders of magnitude faster and use two orders of magnitude less memory, whilst preserving the same average accuracy. This paper is under consideration for acceptance in TPLP.