As a result, Wikipedia articles become the best bet for most users when searching answers for factual queries on the Web #REFR .

Information acquisition is one of the fundamental daily needs of human beings.

Acquiring information from the Web is undoubtedly a convenient and efficient way.

However, with the exponential growth of the Web, information on the Web becomes scattered and evolves quickly, making it challenging for users to acquire the expected information quickly.

The reason is that Wikipedia articles provide credible content in which most claims can be supported by references from reputable sources.

While Wikipedia is a good source of answers for factual queries, the need for manual editing (crowd-sourcing and editor checking) curbs its growth of coverage on a broader range of information needs. What if Wikipedia articles could be automatically generated?

In this paper, we introduce a new task, WEBBRAIN, exploring the capacity of generating short factual articles for queries via a large web corpus.

Given a factual query, the goal of the task is to enable a system to mine supporting evidence from the Web and generate a short factual article in which the claims are supported by the mined evidence (defined in Section 3.1).

One of the potential generation targets for WEBBRAIN is the first section of a new Wiki page, based on which we can further explore generating long factual articles (e.g., a complete Wiki page).

Webbrain: Learning To Generate Factually Correct Articles For Queries By Grounding On Large Web Corpus

Abstract: In this paper, we introduce a new NLP task-generating short factual articles with references for queries by mining supporting evidence from the Web. In this task, called WEBBRAIN, the ultimate goal is to generate a fluent, informative, and factually-correct short article (e.g., a Wikipedia article) for a factual query unseen in Wikipedia. To enable experiments on WEBBRAIN, we construct a large-scale dataset WebBrain-Raw by extracting English Wikipedia articles and their crawlable Wikipedia references. WebBrain-Raw is ten times larger than the previous biggest peer dataset, which can greatly benefit the research community. From WebBrain-Raw, we construct two task-specific datasets: WebBrain-R and WebBrain-G, which are used to train in-domain retriever and generator, respectively. Besides, we empirically analyze the performances of the current state-ofthe-art NLP techniques on WEBBRAIN and introduce a new framework ReGen, which enhances the generation factualness by improved evidence retrieval and task-specific pre-training for generation. Experiment results show that ReGen outperforms all baselines in both automatic and human evaluations. Our code and datasets are released at https://github.com/qhjqhj00/WebBrain.

Why We Read Wikipedia

Abstract: Wikipedia is one of the most popular sites on the Web, with millions of users relying on it to satisfy a broad range of information needs every day. Although it is crucial to understand what exactly these needs are in order to be able to meet them, little is currently known about why users visit Wikipedia. The goal of this paper is to fill this gap by combining a survey of Wikipedia readers with a log-based analysis of user activity. Based on an initial series of user surveys, we build a taxonomy of Wikipedia use cases along several dimensions, capturing users' motivations to visit Wikipedia, the depth of knowledge they are seeking, and their knowledge of the topic of interest prior to visiting Wikipedia. Then, we quantify the prevalence of these use cases via a large-scale user survey conducted on live Wikipedia with almost 30,000 responses. Our analyses highlight the variety of factors driving users to Wikipedia, such as current events, media coverage of a topic, personal curiosity, work or school assignments, or boredom. Finally, we match survey responses to the respondents' digital traces in Wikipedia's server logs, enabling the discovery of behavioral patterns associated with specific use cases. For instance, we observe long and fast-paced page sequences across topics for users who are bored or exploring randomly, whereas those using Wikipedia for work or school spend more time on individual articles focused on topics such as science. Our findings advance our understanding of reader motivations and behavior on Wikipedia and can have implications for developers aiming to improve Wikipedia's user experience, editors striving to cater to their readers' needs, third-party services (such as search engines) providing access to Wikipedia content, and researchers aiming to build tools such as recommendation engines.


Application In Multi-Armed Bandit Problem

Many previous UCB algorithms based on non-asymptotic inference in the literature assume that the sub-G parameter is a preset constant, see the algorithm in #REFR for instance.

(2022) ), so Algorithm 1 can be seen as an optimal algorithm.

Compared with the traditional vanilla UCB, we do improve the constant.

When Y k ∼ N (µ k , 1), the constant factor in regret bound in #OTHEREFR

(2002) is 256, which is larger than 16(2 + √ 2) 2 in our theorem.

When the UCB has unknown sub-G parameters, Theorem 4 first studies a feasible UCB algorithm with sub-G parameter plugging estimation.

Next, we give an simulation for Theorem 4 in two sub-G cases to verify the performance of estimated norms. Similar to #OTHEREFR

(2019) ; , we design the three methods as follows:

Use our method ϕ(Y T k (t) ) with Estimated Norm in Theorem 4;

Tight Non-Asymptotic Inference Via Sub-Gaussian Intrinsic Moment Norm

Abstract: In non-asymptotic statistical inferences, variance-type parameters of sub-Gaussian distributions play a crucial role. However, direct estimation of these parameters based on the empirical moment generating function (MGF) is infeasible. To this end, we recommend using a sub-Gaussian intrinsic moment norm [Buldygin and Kozachenko (2000), Theorem 1.3] through maximizing a series of normalized moments. Importantly, the recommended norm can not only recover the exponential moment bounds for the corresponding MGFs, but also lead to tighter Hoeffiding's sub-Gaussian concentration inequalities. In practice, we propose an intuitive way of checking sub-Gaussian data with a finite sample size by the sub-Gaussian plot. Intrinsic moment norm can be robustly estimated via a simple plug-in approach. Our theoretical results are applied to non-asymptotic analysis, including the multi-armed bandit.

Bootstrapping Upper Confidence Bound

Abstract: Upper Confidence Bound (UCB) method is arguably the most celebrated one used in online decision making with partial information feedback. Existing techniques for constructing confidence bounds are typically built upon various concentration inequalities, which thus lead to over-exploration. In this paper, we propose a non-parametric and data-dependent UCB algorithm based on the multiplier bootstrap. To improve its finite sample performance, we further incorporate second-order correction into the above construction. In theory, we derive both problem-dependent and problem-independent regret bounds for multi-armed bandits under a much weaker tail assumption than the standard sub-Gaussianity. Numerical results demonstrate significant regret reductions by our method, in comparison with several baselines in a range of multi-armed and linear bandit problems.


There thus has been a growing interest in developing parameter-efficient methods for model tuning #REFR , where the goal is to learn only a small number of additional parameters per task while achieving performance comparable to full finetuning.

Finetuning pretrained language models (PLMs) has led to significant improvements across various downstream NLP tasks #OTHEREFR .

However, the conventional paradigm of full task-specific finetuning (FT) is difficult to scale to multiple tasks, given that modern PLMs can have hundreds of millions (or even billions) of parameters.

Prompt tuning (PT), which prepends tunable continuous prompt vectors to the input, has emerged as a promising approach for parameter-efficient transfer learning with PLMs #OTHEREFR .

PT freezes the PLM parameters and only learns a small set of task-specific prompt vectors.

However, despite their impressive performance, there is still a large gap between prompt tuning and full finetuning .

Additionally, this approach is sensitive to initialization and often requires more training time than finetuning #OTHEREFR .

Recent work has proposed to address these issues by transferring prompt vectors from various tasks #OTHEREFR .

Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning

Abstract: Prompt tuning, in which a base pretrained model is adapted to each task via conditioning on learned prompt vectors, has emerged as a promising approach for efficiently adapting large language models to multiple downstream tasks. However, existing methods typically learn soft prompt vectors from scratch, and it has not been clear how to exploit the rich cross-task knowledge with prompt vectors in a multitask learning setting. We propose multitask prompt tuning (MPT), which first learns a single transferable prompt by distilling knowledge from multiple task-specific source prompts. We then learn multiplicative low rank updates to this shared prompt to efficiently adapt it to each downstream target task. Extensive experiments on 23 NLP datasets demonstrate that our proposed approach outperforms the state-of-the-art methods, including the full finetuning baseline in some cases, despite only tuning 0.035% as many task-specific parameters. 1 * Work done during an internship at MIT-IBM Watson AI Lab.

Parameter-Efficient Transfer Learning For Nlp

Abstract: Fine-tuning large pre-trained models is an effective transfer mechanism in NLP. However, in the presence of many downstream tasks, fine-tuning is parameter inefficient: an entire new model is required for every task. As an alternative, we propose transfer with adapter modules. Adapter modules yield a compact and extensible model; they add only a few trainable parameters per task, and new tasks can be added without revisiting previous ones. The parameters of the original network remain fixed, yielding a high degree of parameter sharing. To demonstrate adapter's effectiveness, we transfer the recently proposed BERT Transformer model to 26 diverse text classification tasks, including the GLUE benchmark. Adapters attain near state-of-the-art performance, whilst adding only a few parameters per task. On GLUE, we attain within 0.4% of the performance of full fine-tuning, adding only 3.6% parameters per task. By contrast, fine-tuning trains 100% of the parameters per task.


Our model is also deeply related to the socalled hidden-manifold model #REFR , which has been used as an analytically solvable model of feedforward neural networks fitting datapoints that live on a low dimensional sub-manifold of their embedding space.

Most theoretical studies of (generalized) HMs assume simple distributions for the patterns [2, 3] , while in prac- * Corresponding author; matteo.negri@uniroma1.it tical applications the patterns are linearly or non-linearly encoded from and decoded to a different space [16] .

In this work, we addressed this limitation by proposing a generative model for the patterns where each pattern is produced by the linear combinations of a fixed vocabulary of what we call factors weighted by pattern specific coefficients, followed by an elementwise non-linearity.

We analyze the model in the high-dimensional regime using the replica method for the statistical physics of disordered systems.

This data-generating process generalizes the structure of linear superposition proposed in [17] , where it was discussed in relation to the mapping between a Hopfield network and a restricted Boltzmann machine.

A similar linear (but dense) mapping has been discussed in [18, #OTHEREFR .

In fact, this lowdimensional latent structure is typical of many real-world datasets, e.g. the ones made of natural images.

Here we do not modify the Hebb rule, as we will see that it is enough to produce a new behaviour of the model, in conjunction with the structure of correlation that we choose.

In fact, we observe that if the correlations in the data are strong enough the model switches from a storage phase to a learning phase, in the sense that attractors appear corresponding to the factors in the data.

We argue that this behaviour opens up a new paradigm for this model and shows that it may have some phenomenology in common with neural networks.

The Hidden-Manifold Hopfield Model And A Learning Phase Transition

Abstract: The Hopfield model has a long-standing tradition in statistical physics, being one of the few neural networks for which a theory is available. Extending the theory of Hopfield models for correlated data could help understand the success of deep neural networks, for instance describing how they extract features from data. Motivated by this, we propose and investigate a generalized Hopfield model that we name Hidden-Manifold Hopfield Model : we generate the couplings from P = αN examples with the Hebb rule using a non-linear transformation of D = αDN random vectors that we call factors, with N the number of neurons. Using the replica method, we obtain a phase diagram for the model that shows a phase transition where the factors hidden in the examples become attractors of the dynamics; this phase exists above a critical value of α and below a critical value of αD. We call this behaviour learning transition.

Modelling The Influence Of Data Structure On Learning In Neural Networks: The Hidden Manifold Model

Abstract: The lack of crisp mathematical models that capture the structure of real-world data sets is a major obstacle to the detailed theoretical understanding of deep neural networks. Here, we introduce a generative model for data sets that we call the hidden manifold model (HMM). The idea is to have high-dimensional inputs lie on a lower-dimensional manifold, with labels that depend only on their position within this manifold, akin to a single layer decoder or generator in a generative adversarial network. We first demonstrate the effect of structured data sets by experimentally comparing the dynamics and the performance of two-layer neural networks trained on three different data sets: (i) an unstructured synthetic data set containing random i.i.d. inputs, (ii) a structured data set drawn from the HMM and (iii) a simple canonical data set containing MNIST images. We pinpoint two phenomena related to the dynamics of the networks and their ability to generalise that only appear when training on structured data sets, and we experimentally demonstrate that training networks on data sets drawn from the HMM reproduces both the phenomena seen during training on real dataset. Our main theoretical result is that we show that the learning dynamics in the hidden manifold model is amenable to an analytical treatment by proving a "Gaussian Equivalence Theorem", opening the way to further detailed theoretical studies. In particular, we show how the dynamics of stochastic gradient descent for a two-layer network is captured by a set of ordinary differential equations that track the generalisation error at all times.

socalled hidden-manifold model
feedforward neural networks


CODEGNN #REFR is a graph neural network (GNN) that utilizes both sequential and AST representation of source code.

It is possible that this status quo is caused by the lack of information about the performance of optimizers in ML4SE tasks, as it is unclear how to infer the optimizer effectiveness for the ML4SE domain.

This motivated us to study the efficiency of various optimizers on various ML4SE problems.

CODE2SEQ #OTHEREFR and TREELSTM #OTHEREFR are recurrent neural networks (RNNs) that use Abstract Syntax Trees (AST) together with the source code itself.

Both models are loosely based on the LSTM model, but CODE2SEQ derives a path-based representation of code from the AST, while TREELSTM uses the AST information as is.

Finally, CODETRANSFORMER #OTHEREFR is an augmented transformer model for code, which combines information from multiple relations between tokens to calculate attention.

To test these models, we use two code-to-text generation problems as benchmarks: documentation generation and method name generation.

We chose these two ML4SE related problems to check the consistency of the optimizer performance.

Moreover, these two problems are often used as the benchmarking problems for ML4SE models #OTHEREFR , #OTHEREFR , #OTHEREFR - #OTHEREFR .

For the documentation generation problem, we use the Python and Java parts of the CodeXGLUE dataset #OTHEREFR (JAVA-CODEXGLUE, PYTHON-CODEXGLUE), thus checking whether the programming language of the processed source code affects the optimizer performance.

Judging Adam: Studying The Performance Of Optimization Methods On Ml4Se Tasks

Abstract: Solving a problem with a deep learning model requires researchers to optimize the loss function with a certain optimization method. The research community has developed more than a hundred different optimizers, yet there is scarce data on optimizer performance in various tasks. In particular, none of the benchmarks test the performance of optimizers on source code-related problems. However, existing benchmark data indicates that certain optimizers may be more efficient for particular domains. In this work, we test the performance of various optimizers on deep learning models for source code and find that the choice of an optimizer can have a significant impact on the model quality, with up to twofold score differences between some of the relatively well-performing optimizers. We also find that RAdam optimizer (and its modification with the Lookahead envelope) is the best optimizer that almost always performs well on the tasks we consider. Our findings show a need for a more extensive study of the optimizers in code-related tasks, and indicate that the ML4SE community should consider using RAdam instead of Adam as the default optimizer for coderelated deep learning tasks.

Improved Code Summarization Via A Graph Neural Network

Abstract: Automatic source code summarization is the task of generating natural language descriptions for source code. Automatic code summarization is a rapidly expanding research area, especially as the community has taken greater advantage of advances in neural network and AI technologies. In general, source code summarization techniques use the source code as input and outputs a natural language description. Yet a strong consensus is developing that using structural information as input leads to improved performance. The first approaches to use structural information flattened the AST into a sequence. Recently, more complex approaches based on random AST paths or graph neural networks have improved on the models using flattened ASTs. However, the literature still does not describe the using a graph neural network together with source code sequence as separate inputs to a model. Therefore, in this paper, we present an approach that uses a graph-based neural architecture that better matches the default structure of the AST to generate these summaries. We evaluate our technique using a data set of 2.1 million Java method-comment pairs and show improvement over four baseline techniques, two from the software engineering literature, and two from machine learning literature.


Let {x k } k≥0 be the iterates produced by SGD (Algorithm in (6)), where stochastic gradients are unbiased (i.e., satisfy #REFR ).

The main building block of the unified analysis from #OTHEREFR is the following parametric assumption on the gradient estimate g k and the problem itself. Assumption 1 (Assumption 4.1 from #OTHEREFR ).

Assume that there exist non-negative constants A, B, C, D 1 , D 2 ≥ 0, ρ ∈ (0, 1] and a (possibly) random non-negative sequence {σ #OTHEREFR k } k≥0 such that the following two relations hold

The above assumption is motivated by the analysis of different SGD-type methods and can be derived for standard setups.

The simplest example is Gradient Descent (GD) applied to the minimization problem of L-smooth function f .

Unified Analysis Of Sgd-Type Methods

Abstract: This note focuses on a simple approach to the unified analysis of SGD-type methods from [21] for strongly convex smooth optimization problems. The similarities in the analyses of different stochastic first-order methods are discussed along with the existing extensions of the framework. The limitations of the analysis and several alternative approaches are mentioned as well.

On Biased Compression For Distributed Learning

Abstract: In the last few years, various communication compression techniques have emerged as an indispensable tool helping to alleviate the communication bottleneck in distributed learning. However, despite the fact {\em biased} compressors often show superior performance in practice when compared to the much more studied and understood {\em unbiased} compressors, very little is known about them. In this work we study three classes of biased compression operators, two of which are new, and their performance when applied to (stochastic) gradient descent and distributed (stochastic) gradient descent. We show for the first time that biased compressors can lead to linear convergence rates both in the single node and distributed settings. Our {\em distributed} SGD method enjoys the ergodic rate $\mathcal{O}\left(\frac{\delta L \exp(-K) }{\mu} + \frac{(C + D)}{K\mu}\right)$, where $\delta$ is a compression parameter which grows when more compression is applied, $L$ and $\mu$ are the smoothness and strong convexity constants, $C$ captures stochastic gradient noise ($C=0$ if full gradients are computed on each node) and $D$ captures the variance of the gradients at the optimum ($D=0$ for over-parameterized models). Further, via a theoretical study of several synthetic and empirical distributions of communicated gradients, we shed light on why and by how much biased compressors outperform their unbiased variants. Finally, we propose a new highly performing biased compressor---combination of Top-$k$ and natural dithering---which in our experiments outperforms all other compression techniques.


The images of EgoNeRF are synthesized by applying the volume rendering equation along the camera ray #REFR and the optional environment map.

Specifically, the points x i = o + t i d along the camera ray from camera position o and ray direction d are accumulated to find the pixel value bŷ

C = N i=1 τ i (1 − e −σ(xi)δi )c(x i , d) + τ N +1 c env (d). (11) N = N c + N f

is the number of samples as described in Sec. 4.1.

σ(x) and c(x, d) are obtained from our balanced feature grids in Eq. (5).

Since the size of our feature grid is exponentially increasing along the r direction, we distribute N c coarse samples exponentially rather than uniformly. The second term in Eq. 11is fetched from the environment map

Balanced Spherical Grid For Egocentric View Synthesis

Abstract: Figure 1. We propose a practical solution to reconstruct large-scale scenes from a short egocentric video. (a) Our scalable capturing setup observes the holistic environment by casually swiping a selfie stick with an omnidirectional camera attached. (b) Then we optimize our balanced spherical feature grids which are tailored for the outward-looking setup.

Nerf: Representing Scenes As Neural Radiance Fields For View Synthesis

Abstract: We present a method that achieves state-of-the-art results for synthesizing novel views of complex scenes by optimizing an underlying continuous volumetric scene function using a sparse set of input views. Our algorithm represents a scene using a fully-connected (non-convolutional) deep network, whose input is a single continuous 5D coordinate (spatial location $(x,y,z)$ and viewing direction $(\theta, \phi)$) and whose output is the volume density and view-dependent emitted radiance at that spatial location. We synthesize views by querying 5D coordinates along camera rays and use classic volume rendering techniques to project the output colors and densities into an image. Because volume rendering is naturally differentiable, the only input required to optimize our representation is a set of images with known camera poses. We describe how to effectively optimize neural radiance fields to render photorealistic novel views of scenes with complicated geometry and appearance, and demonstrate results that outperform prior work on neural rendering and view synthesis. View synthesis results are best viewed as videos, so we urge readers to view our supplementary video for convincing comparisons.


In support of this review of prior work on racial bias mitigation a summary table of related work is provided to compare overall relative performance on the RFW dataset #REFR (Table 3 ).

where is the upper bound of the penalty interval, and is the number of feature representation vectors pairs whose cosine similarity lies within the interval (0, ). domains.

In another example in face recognition, #OTHEREFR introduces Cross-Domain Triplet (CDT) loss based on the triplet loss #OTHEREFR and uses similarity metrics from one domain to learn compact feature clusters of identities by incorporating them into another domain.

Relative performance for both CDT and MFR on the RFW dataset are shown in Table 3 .

This section presents a brief overview of face representation learning, including the potential sources of biases and mitigation studies within this stage of the face recognition processing pipeline (Fig. 3) .

Racial Bias Within Face Recognition: A Survey

Abstract: Facial recognition is one of the most academically studied and industrially developed areas within computer vision where we readily find associated applications deployed globally. This widespread adoption has uncovered significant performance variation across subjects of different racial profiles leading to focused research attention on racial bias within face recognition spanning both current causation and future potential solutions. In support, this study provides an extensive taxonomic review of research on racial bias within face recognition exploring every aspect and stage of the face recognition processing pipeline. Firstly, we discuss the problem definition of racial bias, starting with race definition, grouping strategies, and the societal implications of using race or race-related groupings. Secondly, we divide the common face recognition processing pipeline into four stages: image acquisition, face localisation, face representation, face verification and identification, and review the relevant corresponding literature associated with each stage. The overall aim is to provide comprehensive coverage of the racial bias problem with respect to each and every stage of the face recognition processing pipeline whilst also highlighting the potential pitfalls and limitations of contemporary mitigation strategies that need to be considered within future research endeavours or commercial applications alike.

Mitigate Bias In Face Recognition Using Skewness-Aware Reinforcement Learning

Abstract: Racial equality is an important theme of international human rights law, but it has been largely obscured when the overall face recognition accuracy is pursued blindly. More facts indicate racial bias indeed degrades the fairness of recognition system and the error rates on non-Caucasians are usually much higher than Caucasians. To encourage fairness, we introduce the idea of adaptive margin to learn balanced performance for different races based on large margin losses. A reinforcement learning based race balance network (RL-RBN) is proposed. We formulate the process of finding the optimal margins for non-Caucasians as a Markov decision process and employ deep Q-learning to learn policies for an agent to select appropriate margin by approximating the Q-value function. Guided by the agent, the skewness of feature scatter between races can be reduced. Besides, we provide two ethnicity aware training datasets, called BUPT-Globalface and BUPT-Balancedface dataset, which can be utilized to study racial bias from both data and algorithm aspects. Extensive experiments on RFW database show that RL-RBN successfully mitigates racial bias and learns more balanced performance for different races.


UFODT performs even better than VFDT and EFDT on LED, Heart and Fico datasets. 2018] as in #REFR .

We compute the average number of queries per online session to compare the costs of algorithms.

We also evaluate the generalization power of classifiers via holdout test sets.

Additionally, (in the appendix F) we measure the prediction performance on training sets during learning together with various other aspects of our framework. Datasets.

We have used three stationary datasets in our experiments that are standard binary classification datasets taken from UCI repository #OTHEREFR .

Furthermore, we conduct experiments on the ProPublica recidivism (Compas) dataset #OTHEREFR and the Fair Isaac (Fico) credit risk dataset [FICO et al., Figure 2 : Test utilities during training: UFODT reaches test utilities comparable with those from VFDT and EFDT but with significantly lower costs.

In Compas dataset, we predict the individuals arrested after two years of release, and in Fico we predict if an individual will default on a loan.

For concept drifting experiments, we adopt the non-stationary Stagger dataset #OTHEREFR , where each data has three nominal attributes and the target concept will change abruptly at some point.

For extensions to continuous features (as well as for feature selection in the appendix), we use Prima Indians Diabetes Dataset [Smith et al., 1988] Algorithms.

The VFDT algorithm #OTHEREFR ] is used as a classical baseline ODT model.

We also compare our method with the EFDT algorithm #OTHEREFR .

Efficient Online Decision Tree Learning With Active Feature Acquisition

Abstract: Constructing decision trees online is a classical machine learning problem. Existing works often assume that features are readily available for each incoming data point. However, in many real world applications, both feature values and the labels are unknown a priori and can only be obtained at a cost. For example, in medical diagnosis, doctors have to choose which tests to perform (i.e., making costly feature queries) on a patient in order to make a diagnosis decision (i.e., predicting labels). We provide a fresh perspective to tackle this practical challenge. Our framework consists of an active planning oracle embedded in an online learning scheme for which we investigate several information acquisition functions. Specifically, we employ a surrogate information acquisition function based on adaptive submodularity to actively query feature values with a minimal cost, while using a posterior sampling scheme to maintain a low regret for online prediction. We demonstrate the efficiency and effectiveness of our framework via extensive experiments on various real-world datasets. Our framework also naturally adapts to the challenging setting of online learning with concept drift and is shown to be competitive with baseline models while being more flexible.

Near Optimal Bayesian Active Learning For Decision Making

Abstract: How should we gather information to make effective decisions? We address Bayesian active learning and experimental design problems, where we sequentially select tests to reduce uncertaintyaboutasetofhypotheses. Instead ofminimizinguncertaintyperse,weconsidera set of overlappingdecision regions of these hypotheses. Our goal is to drive uncertainty into a single decision region as quickly as possible. We identify necessary and sucient conditionsforcorrectlyidentifyingadecisionregion that contains all hypotheses consistent with observations. We develop a novel Hyperedge Cutting (HEC) algorithm for this problem, and prove that is competitive with the intractable optimal policy. Our ecient implementation of the algorithm relies on computingsubsetsofthecompletehomogeneoussymmetric polynomials. Finally, we demonstrate its eectiveness on two practical applications: approximate comparison-based learning and activelocalizationusingarobotmanipulator.


C.1 Analysis Of The Relationship Between Gnn Size And Error

Second, the training error does not approach zero, indicating that even a large model is not over-parameterized #REFR .

This inequality suggests that smaller model sizes typically lead to lower test errors when training a model from scratch.

This forms the basis for the effectiveness of our delta tuning method.

We conducted experiments on six small molecular datasets, varying model sizes in two ways: (1) by varying embedding dimensions while fixing the MLP middle dimension at twice the embedding dimensions, and (2) by fixing embedding dimensions at 300 and varying the middle dimensions.

The results are presented in Figures 6 and 7 . For all datasets, we made two observations.

First, the test error initially decreases and then increases as the model size grows.

These observations suggest that our tasks fall within the scope of the classical regime and reducing the size of the parameter space, unless too small, can help improve generalization ability.

It is important to note that the phenomenon in question occurs exclusively in the context of training from scratch.

To leverage the knowledge gained from pre-training, employing a larger model is necessary.

To enhance the generalization ability of this large model, the full fine-tuning approach can be replaced with delta tuning. As illustrated in Fig.

3(a) , embedding dimensions between 200 to 300 exhibit the highest performance for full fine-tuning, albeit with inferior test error when training from scratch.

Adaptergnn: Efficient Delta Tuning Improves Generalization Ability In Graph Neural Networks

Abstract: Fine-tuning pre-trained models has recently yielded remarkable performance gains in graph neural networks (GNNs). In addition to pre-training techniques, inspired by the latest work in the natural language fields, more recent work has shifted towards applying effective fine-tuning approaches, such as parameter-efficient tuning (delta tuning). However, given the substantial differences between GNNs and transformer-based models, applying such approaches directly to GNNs proved to be less effective. In this paper, we present a comprehensive comparison of delta tuning techniques for GNNs and propose a novel delta tuning method specifically designed for GNNs, called AdapterGNN. AdapterGNN preserves the knowledge of the large pre-trained model and leverages highly expressive adapters for GNNs, which can adapt to downstream tasks effectively with only a few parameters, while also improving the model's generalization ability on the downstream tasks. Extensive experiments show that AdapterGNN achieves higher evaluation performance (outperforming full fine-tuning by 1.4% and 5.5% in the chemistry and biology domains respectively, with only 5% of its parameters tuned) and lower generalization gaps compared to full fine-tuning. Moreover, we empirically show that a larger GNN model can have a worse generalization ability, which differs from the trend observed in large language models. We have also provided a theoretical justification for delta tuning can improve the generalization ability of GNNs by applying generalization bounds.

Deep Double Descent: Where Bigger Models And More Data Hurt

Abstract: We show that a variety of modern deep learning tasks exhibit a "double-descent" phenomenon where, as we increase model size, performance first gets worse and then gets better. Moreover, we show that double descent occurs not just as a function of model size, but also as a function of the number of training epochs. We unify the above phenomena by defining a new complexity measure we call the effective model complexity and conjecture a generalized double descent with respect to this measure. Furthermore, our notion of model complexity allows us to identify certain regimes where increasing (even quadrupling) the number of train samples actually hurts test performance.


Further, they proposed to learn "flat manifolds" with VAEs #REFR , whose decoder essentially preserves the Euclidean distance between points in the latent space and the decoded points in the data space.

The encoders in commonly used traditional codecs such as H.264 #OTHEREFR and HEVC #OTHEREFR are also equipped with an exhaustive search procedure to select the optimal block partitioning and coding modes for image frame.

More recently, the idea of iterative and optimization-based encoding is becoming increasingly prominent in nonlinear transform coding #OTHEREFR , as well as computer vision in the form of implicit neural representations #OTHEREFR .

It is therefore interesting to see whether ideas from vector quantization and implicit neural representations may prove fruitful for further reducing the decoding complexity in NTC.

Manifold learning A distantly related line of work is in metric learning with deep generative models, where the idea is to learn a latent representation of the data such that distance in the latent space preserves the similarity in the data space. Chen et al.

#OTHEREFR proposes the use of the Riemannian distance metric induced by a decoding transform of a latent variable to measure similarity in the data space.

Their method is based on regularizing the Jacobian of the VAE to be constant, essentially resulting in a linear (affine) decoder with similar behavior to what we observed in learned synthesis transforms in Section 3.1.

Asymmetrically-Powered Neural Image Compression With Shallow Decoders

Abstract: Neural image compression methods have seen increasingly strong performance in recent years. However, they suffer orders of magnitude higher computational complexity compared to traditional codecs, which stands in the way of real-world deployment. This paper takes a step forward in closing this gap in decoding complexity by adopting shallow or even linear decoding transforms. To compensate for the resulting drop in compression performance, we exploit the often asymmetrical computation budget between encoding and decoding, by adopting more powerful encoder networks and iterative encoding. We theoretically formalize the intuition behind, and our experimental results establish a new frontier in the trade-off between rate-distortion and decoding complexity for neural image compression. Specifically, we achieve rate-distortion performance competitive with the established mean-scale hyperprior architecture of Minnen et al. (2018), while reducing the overall decoding complexity by 80 %, or over 90 % for the synthesis transform alone. Our code can be found at https: //github.com/mandt-lab/shallow-ntc. 1

Learning Flat Latent Manifolds With Vaes

Abstract: Measuring the similarity between data points often requires domain knowledge. This can in parts be compensated by relying on unsupervised methods such as latent-variable models, where similarity/distance is estimated in a more compact latent space. Prevalent is the use of the Euclidean metric, which has the drawback of ignoring information about similarity of data stored in the decoder, as captured by the framework of Riemannian geometry. Alternatives---such as approximating the geodesic---are often computationally inefficient, rendering the methods impractical. We propose an extension to the framework of variational auto-encoders allows learning flat latent manifolds, where the Euclidean metric is a proxy for the similarity between data points. This is achieved by defining the latent space as a Riemannian manifold and by regularising the metric tensor to be a scaled identity matrix. Additionally, we replace the compact prior typically used in variational auto-encoders with a recently presented, more expressive hierarchical one---and formulate the learning problem as a constrained optimisation problem. We evaluate our method on a range of data-sets, including a video-tracking benchmark, where the performance of our unsupervised approach nears that of state-of-the-art supervised approaches, while retaining the computational efficiency of straight-line-based approaches.


Although the CDC loss could in principle be applied to encoder features directly, we first project the dense features to a dedicated 128-dimensional embedding space, as per standard practice #REFR .

The cross-domain contrastive (CDC) loss is the central component of CMA.

It incentivizes the encoder to learn features which discriminatively reflect the semantics, but are invariant to the visual condition.

To this end, spatial target image features represent anchors, which are pulled towards spatially corresponding reference image features-the positives.

The anchors and positives are assumed to represent similar semantics in distinct visual conditions.

Simultaneously, the anchors are contrasted to other target image features-the negatives-to prevent mode collapse.

The projection head PROJ consists of two 1×1 convolutions with a ReLU non-linearity inbetween. As shown in Fig.

2 , the embeddings of the trainable model PROJ θ • ENC θ serve as anchors, while-as proposed in #OTHEREFR -positives and negatives are obtained by an exponential moving average model PROJ ema • ENC ema to improve their consistency.

Furthermore, we use a queue to accumulate negatives #OTHEREFR .

This enables the use of a large number of negatives during instance discrimination, which encourages the learning of meaningful representations by making the discrimination more challenging.

Finally, the positives are spatially warped to align them with the anchors, as detailed in Sec. 3.2.

Contrastive Model Adaptation For Cross-Condition Robustness In Semantic Segmentation

Abstract: Standard unsupervised domain adaptation methods adapt models from a source to a target domain using labeled source data and unlabeled target data jointly. In model adaptation, on the other hand, access to the labeled source data is prohibited, i.e., only the source-trained model and unlabeled target data are available. We investigate normal-to-adverse condition model adaptation for semantic segmentation, whereby image-level correspondences are available in the target domain. The target set consists of unlabeled pairs of adverse-and normalcondition street images taken at GPS-matched locations. Our method-CMA-leverages such image pairs to learn condition-invariant features via contrastive learning. In particular, CMA encourages features in the embedding space to be grouped according to their condition-invariant semantic content and not according to the condition under which respective inputs are captured. To obtain accurate cross-domain semantic correspondences, we warp the normal image to the viewpoint of the adverse image and leverage warp-confidence scores to create robust, aggregated features. With this approach, we achieve state-of-the-art semantic segmentation performance for model adaptation on several normal-to-adverse adaptation benchmarks, such as ACDC and Dark Zurich. We also evaluate CMA on a newly procured adverse-condition generalization benchmark and report favorable results compared to standard unsupervised domain adaptation methods, despite the comparative handicap of CMA due to source data inaccessibility. Code is available at https://github.com/brdav/cma.

A Simple Framework For Contrastive Learning Of Visual Representations

Abstract: This paper presents SimCLR: a simple framework for contrastive learning of visual representations. We simplify recently proposed contrastive self-supervised learning algorithms without requiring specialized architectures or a memory bank. In order to understand what enables the contrastive prediction tasks to learn useful representations, we systematically study the major components of our framework. We show that (1) composition of data augmentations plays a critical role in defining effective predictive tasks, (2) introducing a learnable nonlinear transformation between the representation and the contrastive loss substantially improves the quality of the learned representations, and (3) contrastive learning benefits from larger batch sizes and more training steps compared to supervised learning. By combining these findings, we are able to considerably outperform previous methods for self-supervised and semi-supervised learning on ImageNet. A linear classifier trained on self-supervised representations learned by SimCLR achieves 76.5% top-1 accuracy, which is a 7% relative improvement over previous state-of-the-art, matching the performance of a supervised ResNet-50. When fine-tuned on only 1% of the labels, we achieve 85.8% top-5 accuracy, outperforming AlexNet with 100X fewer labels.

dense features
dedicated 128-dimensional embedding


This is in contrast to related works #REFR Chen et al., 2022) that require the full world model during inference. 2.

The Transformer-XL architecture #OTHEREFR is much more computationally efficient than vanilla transformers at inference time and introduces relative positional encodings, which remove the dependence on absolute time steps.

Our contributions: The contributions of this work can be summarized as follows:

We present a new autoregressive world model based on the Transformer-XL #OTHEREFR ) architecture and a model-free agent trained in latent imagination.

Running our policy is computationally efficient, as the transformer is not needed at inference time.

Our world model is provided with information on how much reward has already been emitted by feeding back predicted rewards into the world model. As shown in our ablation study, this improves performance. 3.

We rewrite the balanced KL divergence loss of #OTHEREFR

(2021) to allow us to fine-tune the relative weight of the involved entropy and cross-entropy terms. 4.

We introduce a new thresholded entropy loss that stabilizes the policy's entropy during training and hereby simplifies the selection of hyperparameters that behave well across different games. 5.

We propose a new effective sampling procedure for the growing dataset of experience, which balances the training distribution to shift the focus towards the latest experience.

Transformer-Based World Models Are Happy With 100K Interactions

Abstract: Deep neural networks have been successful in many reinforcement learning settings. However, compared to human learners they are overly data hungry. To build a sample-efficient world model, we apply a transformer to real-world episodes in an autoregressive manner: not only the compact latent states and the taken actions but also the experienced or predicted rewards are fed into the transformer, so that it can attend flexibly to all three modalities at different time steps. The transformer allows our world model to access previous states directly, instead of viewing them through a compressed recurrent state. By utilizing the Transformer-XL architecture, it is able to learn long-term dependencies while staying computationally efficient. Our transformer-based world model (TWM) generates meaningful, new experience, which is used to train a policy that outperforms previous model-free and model-based reinforcement learning algorithms on the Atari 100k benchmark.

Dream To Control: Learning Behaviors By Latent Imagination

Abstract: To select effective actions in complex environments, intelligent agents need to generalize from past experience. World models can represent knowledge about the environment to facilitate such generalization. While learning world models from high-dimensional sensory inputs is becoming feasible through deep learning, there are many potential ways for deriving behaviors from them. We present Dreamer, a reinforcement learning agent that solves long-horizon tasks purely by latent imagination. We efficiently learn behaviors by backpropagating analytic gradients of learned state values through trajectories imagined in the compact state space of a learned world model. On 20 challenging visual control tasks, Dreamer exceeds existing approaches in data-efficiency, computation time, and final performance.


We use DistilBERT #REFR as the pre-trained language model for the first stage models and for reranking Electra-large and T5-3B.

Note that this could be extended to other models, but we chose the three that we had most familiarity with.

In more detail, we use DPR-cls #OTHEREFR as our dense retriever, SPLADE-max #OTHEREFR as our learned sparse retriever and RankT5-3b encoder only #OTHEREFR , monoElectra-large #OTHEREFR as our cross encoding rerankers.

We train models on either the "titled" or the official MS MARCO-passage dataset. Titled passages are generated using "Title [SEP] Passage". We use only contrastive training (i.e.

no distillation) and hard-negatives #OTHEREFR to train the first stage models and the negatives from SPLADE to train the reranker.

First-stage retrieval training uses 8 queries per batch, and 32 negatives per query, and trains for 3 epochs.

Statistical significance is computed only when we directly compare the original vs "title" datasets (i.e.

only Tables 1 and 4 and only on the same model with different corpus), with Student's t-test and ≤ 0.05.

The Tale Of Two Ms Marco -And Their Unfair Comparisons

Abstract: The MS MARCO-passage dataset has been the main large-scale dataset open to the IR community and it has fostered successfully the development of novel neural retrieval models over the years. But, it turns out that two different corpora of MS MARCO are used in the literature, the official one and a second one where passages were augmented with titles, mostly due to the introduction of the Tevatron code base. However, the addition of titles actually leaks relevance information, while breaking the original guidelines of the MS MARCO-passage dataset. In this work, we investigate the differences between the two corpora and demonstrate empirically that they make a significant difference when evaluating a new method. In other words, we show that if a paper does not properly report which version is used, reproducing fairly its results is basically impossible. Furthermore, given the current status of reviewing, where monitoring state-of-the-art results is of great importance, having two different versions of a dataset is a large problem. This is why this paper aims to report the importance of this issue so that researchers can be made aware of this problem and appropriately report their results.

Distilbert, A Distilled Version Of Bert: Smaller, Faster, Cheaper And Lighter

Abstract: As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or inference budgets remains challenging. In this work, we propose a method to pre-train a smaller general-purpose language representation model, called DistilBERT, which can then be fine-tuned with good performances on a wide range of tasks like its larger counterparts. While most prior work investigated the use of distillation for building task-specific models, we leverage knowledge distillation during the pre-training phase and show that it is possible to reduce the size of a BERT model by 40%, while retaining 97% of its language understanding capabilities and being 60% faster. To leverage the inductive biases learned by larger models during pre-training, we introduce a triple loss combining language modeling, distillation and cosine-distance losses. Our smaller, faster and lighter model is cheaper to pre-train and we demonstrate its capabilities for on-device computations in a proof-of-concept experiment and a comparative on-device study.


In the past, many advances have been driven in the field of fluid mechanics #REFR , and this is likely to continue.

Understanding how these solution operators vary with system parameters is an important avenue of ongoing research #OTHEREFR .

Similarly, machine learning may be used to accelerate traditional scientific computing workflows, for example by flexible super-resolution or learning of improved solution stencils.

However, here are several challenges with these approaches, foremost the fact that traditional numerical algorithms are extremely mature and scaleable, so that machine learning solutions are expected to compete with decades of progress.

In all of the cases explored in this perspective, progress will be accelerated by a diverse and robust set of benchmark problems with which to assess new solutions #OTHEREFR .

In addition, we must stress that these techniques are primarily tools to be used by human experts for scientific discovery.

For example, understanding sensitivities with resolvent analysis #OTHEREFR , using physics informed neural networks (PINNs) #OTHEREFR for RANS modeling #OTHEREFR , and using wall measurements to estimate turbulent flow fields #OTHEREFR are all exciting avenues of research.

Interestingly, there are also efforts to understand neural networks using techniques from PDEs #OTHEREFR .

Although there is a desire for automated machine learning algorithms, when applied to science and engineering applications, this is still primarily a human endeavor.

However, progress in the field of PDEs, enabled by machine learning, is undeniable.

Despite this progress, there is still much we don't know about PDEs.

Machine Learning For Partial Differential Equations

Abstract: Partial differential equations (PDEs) are among the most universal and parsimonious descriptions of natural physical laws, capturing a rich variety of phenomenology and multi-scale physics in a compact and symbolic representation. This review will examine several promising avenues of PDE research that are being advanced by machine learning, including: 1) the discovery of new governing PDEs and coarse-grained approximations for complex natural and engineered systems, 2) learning effective coordinate systems and reduced-order models to make PDEs more amenable to analysis, and 3) representing solution operators and improving traditional numerical algorithms. In each of these fields, we summarize key advances, ongoing challenges, and opportunities for further development.

Machine Learning For Fluid Mechanics

Abstract: The field of fluid mechanics is rapidly advancing, driven by unprecedented volumes of data from field measurements, experiments and large-scale simulations at multiple spatiotemporal scales. Machine learning offers a wealth of techniques to extract information from data that could be translated into knowledge about the underlying fluid mechanics. Moreover, machine learning algorithms can augment domain knowledge and automate tasks related to flow control and optimization. This article presents an overview of past history, current developments, and emerging opportunities of machine learning for fluid mechanics. It outlines fundamental machine learning methodologies and discusses their uses for understanding, modeling, optimizing, and controlling fluid flows. The strengths and limitations of these methods are addressed from the perspective of scientific inquiry that considers data as an inherent part of modeling, experimentation, and simulation. Machine learning provides a powerful information processing framework that can enrich, and possibly even transform, current lines of fluid mechanics research and industrial applications.


When the number of observations is high, the "no coupling" baseline #REFR reaches the performances of SDSBM.

4b ., we vary the number of observations available for each item from 100 to 10.000, distributed over 100 epochs.

Thus, in the most challenging situation, there is only one observation per epoch used to determine dynamic memberships over 3 clusters. In the expression of , is set to 0.05. We see Fig. 4b .

that for both patterns, the predictive power of SDSBM remains high in such conditions.

Moreover, the RMSE on the true dynamic memberships in this case is fairly low, and decreases rapidly as the number of observations increases.

This is due to SDSBM linking the time slices together during the training phase: the smoothing constraint makes it so that every time slices indirectly benefits from the training data of its neighbours.

This is because as the number of observations in each slice goes to infinity, the models need less to rely on temporal neighbours.

However, even for 10.000 observations per item (100 observations per epoch), SDSBM yields better results. As an illustration, the results in Fig.

1 have been obtained using only 5 observations per epoch.

4.1.3 SDSBM handles highly stochastic interaction patterns.

Finally, we control the deterministic character of the block-interaction matrix by varying .

Dynamic Mixed Membership Stochastic Block Model For Weighted Labeled Networks

Abstract: Most real-world networks evolve over time. Existing literature proposes models for dynamic networks that are either unlabeled or assumed to have a single membership structure. On the other hand, a new family of Mixed Membership Stochastic Block Models (MMSBM) allows to model static labeled networks under the assumption of mixed-membership clustering. In this work, we propose to extend this later class of models to infer dynamic labeled networks under a mixed membership assumption. Our approach takes the form of a temporal prior on the model's parameters. It relies on the single assumption that dynamics are not abrupt. We show that our method significantly differs from existing approaches, and allows to model more complex systems-dynamic labeled networks. We demonstrate the robustness of our method with several experiments on both synthetic and real-world datasets. A key interest of our approach is that it needs very few training data to yield good results. The performance gain under challenging conditions broadens the variety of possible applications of automated learning tools-as in social sciences, which comprise many fields where small datasets are a major obstacle to the introduction of machine learning methods.

Tensorial And Bipartite Block Models For Link Prediction In Layered Networks And Temporal Networks

Abstract: Many real-world complex systems are well represented as multilayer networks; predicting interactions in those systems is one of the most pressing problems in predictive network science. To address this challenge, we introduce two stochastic block models for multilayer and temporal networks; one of them uses nodes as its fundamental unit, whereas the other focuses on links. We also develop scalable algorithms for inferring the parameters of these models. Because our models describe all layers simultaneously, our approach takes full advantage of the information contained in the whole network when making predictions about any particular layer. We illustrate the potential of our approach by analyzing two empirical datasets---a temporal network of email communications, and a network of drug interactions for treating different cancer types. We find that modeling all layers simultaneously does result, in general, in more accurate link prediction. However, the most predictive model depends on the dataset under consideration; whereas the node-based model is more appropriate for predicting drug interactions, the link-based model is more appropriate for predicting email communication.


According to #REFR , we work on inferring the newly defined effective reaction rate k f = v A k f,r as well as a r .

where C A and C B are the concentration of solutes A and B respectively, q = 0.5 is the Darcy velocity, ψ = 0.4 is the porosity, D = 1 × 10 −8 is the diffusion coefficient, v A = a r and v B = −1 are the stoichiometric coefficients.

The constant k f,r is the chemical reaction rate and a r is the order of the chemical reaction, both of which are difficult to measure and will be inferred by the observations of C A .

To obtain the observations, numerical simulations are conducted using a secondorder finite difference method for spatial discretization and the second-order backward difference method for implicitly marching in time.

In simulations, the mesh size is ∆x = 0.0125 and the time step size is ∆t = 0.005.

We set the exact values to k f = 1.577 and a r = 2. The high-fidelity observations at Table 3 .

It is shown that the inferred results are highly accurate with a small relative error and variance.

Our results are significantly outperforming the reported results in #OTHEREFR .

It is worth noting that only 50 residual points are used for training the networks, while 30000 residual points are used in #OTHEREFR .

In order to take a further look at the prediction details, Fig.

Feature-Adjacent Multi-Fidelity Physics-Informed Machine Learning For Partial Differential Equations

Abstract: Physics-informed neural networks have emerged as an alternative method for solving partial differential equations. However, for complex problems, the training of such networks can still require high-fidelity data which can be expensive to generate. To reduce or even eliminate the dependency on high-fidelity data, we propose a novel multi-fidelity architecture which is based on a feature space shared by the lowand high-fidelity solutions. In the feature space, the projections of the low-fidelity and high-fidelity solutions are adjacent by constraining their relative distance. The feature space is represented with an encoder and its mapping to the original solution space is effected through a decoder. The proposed multi-fidelity approach is validated on forward and inverse problems for steady and unsteady problems described by partial differential equations.

A Composite Neural Network That Learns From Multi-Fidelity Data: Application To Function Approximation And Inverse Pde Problems

Abstract: Currently the training of neural networks relies on data of comparable accuracy but in real applications only a very small set of high-fidelity data is available while inexpensive lower fidelity data may be plentiful. We propose a new composite neural network (NN) that can be trained based on multi-fidelity data. It is comprised of three NNs, with the first NN trained using the low-fidelity data and coupled to two high-fidelity NNs, one with activation functions and another one without, in order to discover and exploit nonlinear and linear correlations, respectively, between the low-fidelity and the high-fidelity data. We first demonstrate the accuracy of the new multi-fidelity NN for approximating some standard benchmark functions but also a 20-dimensional function that is not easy to approximate with other methods, e.g. Gaussian process regression. Subsequently, we extend the recently developed physics-informed neural networks (PINNs) to be trained with multi-fidelity data sets (MPINNs). MPINNs contain four fully-connected neural networks, where the first one approximates the low-fidelity data, while the second and third construct the correlation between the low- and high-fidelity data and produce the multi-fidelity approximation, which is then used in the last NN that encodes the partial differential equations (PDEs). Specifically, by decomposing the correlation into a linear and nonlinear part, the present model is capable of learning both the linear and complex nonlinear correlations between the low- and high-fidelity data adaptively. By training the MPINNs, we can: (1) obtain the correlation between the low- and high-fidelity data, (2) infer the quantities of interest based on a few scattered data, and (3) identify the unknown parameters in the PDEs. In particular, we employ the MPINNs to learn the hydraulic conductivity field for unsaturated flows as well as the reactive models for reactive transport. The results demonstrate that MPINNs can achieve relatively high accuracy based on a very small set of high-fidelity data. Despite the relatively low dimension and limited number of fidelities (two-fidelity levels) for the benchmark problems in the present study, the proposed model can be readily extended to very high-dimensional regression and classification problems involving multi-fidelity data.


To further explore the number of word pieces per entity type, we isolate the unique entities #REFR .

We choose the biomedical (ADE) dataset to explore the effect of tokenization in a special domain.

The ADE dataset contains entities of Drugs and Adverse Effects (AE) and has labels for the relations between them.

The tokenizer of cased BERT #OTHEREFR and bioclinical BERT (b-BERT) #OTHEREFR is based on the WordPiece algorithm, while ALBERT #OTHEREFR adopts the SentencePiece algorithm. In Tab.

2, we present the effect of tokenization on the average sentence length, in terms of word pieces (subwords), for each dataset.

The sentence length increases by approximately 12 tokens, up to 58%, after the tokenization in the biomedical domain.

Then, we find the unique words that are part of each entity type and tokenize the unique entities and words using the different tokenizers to notice the difference in the length and the addition of the word pieces. In Tab.

1 the last column represents the average tokenized word length per entity type, and the Out type describes the words that are not part of an entity of interest.

In the ADE dataset, the length of the drug and AE entities increases substantially, and the drug entities are split into more word pieces.

Particularly, a word that is part of a drug entity is split into approximately 4 word pieces, on average, when using the tokenizer of cased BERT and b-BERT.

The tokenizer of ALBERT tends to split the entities of interest into fewer pieces.

An Information Extraction Study: Take In Mind The Tokenization!

Abstract: Current research on the advantages and trade-offs of using characters, instead of tokenized text, as input for deep learning models, has evolved substantially. New token-free models remove the traditional tokenization step; however, their efficiency remains unclear. Moreover, the effect of tokenization is relatively unexplored in sequence tagging tasks. To this end, we investigate the impact of tokenization when extracting information from documents and present a comparative study and analysis of subword-based and character-based models. Specifically, we study Information Extraction (IE) from biomedical texts. The main outcome is twofold: tokenization patterns can introduce inductive bias that results in state-of-the-art performance, and the character-based models produce promising results; thus, transitioning to token-free IE models is feasible.

Arabert: Transformer-Based Model For Arabic Language Understanding

Abstract: The Arabic language is a morphologically rich language with relatively few resources and a less explored syntax compared to English. Given these limitations, Arabic Natural Language Processing (NLP) tasks like Sentiment Analysis (SA), Named Entity Recognition (NER), and Question Answering (QA), have proven to be very challenging to tackle. Recently, with the surge of transformers based models, language-specific BERT based models have proven to be very efficient at language understanding, provided they are pre-trained on a very large corpus. Such models were able to set new standards and achieve state-of-the-art results for most NLP tasks. In this paper, we pre-trained BERT specifically for the Arabic language in the pursuit of achieving the same success that BERT did for the English language. The performance of AraBERT is compared to multilingual BERT from Google and other state-of-the-art approaches. The results showed that the newly developed AraBERT achieved state-of-the-art performance on most tested Arabic NLP tasks. The pretrained araBERT models are publicly available on https://github.com/aub-mind/arabert hoping to encourage research and applications for Arabic NLP.


The 2D branch processes images to obtain a pixel-wise prediction given x 2D and it consists of a standard 2D U-Net #REFR .

We build our contributions upon the two independent branches (2D and 3D) architecture proposed in #OTHEREFR .

On the other hand, the 3D branch takes in input point clouds to estimate the class of each point of x 3D and it is implemented as a 3D sparse convolutional network #OTHEREFR .

Thanks to the fact that 2D-3D correspondences are known, 3D points can be projected into the image plane to supervise the 2D branch, as supervision is provided only for the sparse 3D points.

We denote the 3D semantic labels projected into 2D with the symbol y 3D→2D .

As argued by #OTHEREFR , such design choice allows one to take advantage of the strengths of each input modality, and final predictions can be obtained by averaging the outputs of the two branches to achieve an effective ensemble.

In our work, we adopt the same framework, and we give an intuitive explanation of why this design choice is particularly effective.

Exploiting The Complementarity Of 2D And 3D Networks To Address Domain-Shift In 3D Semantic Segmentation

Abstract: 2D Receptive Field Re-Projected into 3D Figure 1. 3D (top) and 2D (bottom) networks processing point clouds and images of the same scene extract features that contain complementary information. Indeed, 2D and 3D effective receptive fields [34] centered on a point n focus on different portions of the scene, i.e., 2D or 3D neighborhoods respectively. Thus, corresponding features have different content by construction. We exploit this property to reduce the domain gap in 3D semantic segmentation.

U-Net: Convolutional Networks For Biomedical Image Segmentation

Abstract: There is large consent that successful training of deep networks requires many thousand annotated training samples. In this paper, we present a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently. The architecture consists of a contracting path to capture context and a symmetric expanding path that enables precise localization. We show that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks. Using the same network trained on transmitted light microscopy images (phase contrast and DIC) we won the ISBI cell tracking challenge 2015 in these categories by a large margin. Moreover, the network is fast. Segmentation of a 512x512 image takes less than a second on a recent GPU. The full implementation (based on Caffe) and the trained networks are available at http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net .


Sampling From The Von Mises-Fisher Distribution

The gradient for this sampling procedure can be computed using the reparameterization trick for acceptance-rejection sampling schemes as proposed by #REFR and further defined for the vMF distribution by .

The next step is to construct a Householder reflection matrix H, defined as H = I − 2hh T , where H = I is the identity matrix and h = e1−µ e1−µ , with modal vector e 1 = (1, 0, • • • , 0).

Applying this Householder transform to z essentially reflects it across the hyperplane that lies between µ and e 1 , resulting in z = Hz , a direction vector sampled from the vMF distribution.

Algorithm 1 vMF Sampling 1: input: dimension m, mean µ, concentration κ 2: Acceptance-rejection sampling:

ω ∼ g(ω | κ, m) ∝ exp(ωκ) 1 − ω 2 1 2 (m−3) 3: Sample v from Uniform distribution: v ∼ U S m−2 4: Householder transform: z ← ω; √ 1 − ω 2 v H ← Householder (e 1 , µ) 5: return : Hz

Geometry-Aware Latent Representation Learning For Modeling Disease Progression Of Barrett'S Esophagus

Abstract: First and foremost, I would like to thank my supervisors, Erik Bekkers, Onno de Boer and Sybren Meijer. Erik for his valuable supervision and feedback, and for always supplying me with many interesting research ideas. His insights introduced me to many new ideas that have also fueled my enthusiasm for research in general. Furthermore, Onno and Sybren for introducing me to the field of histopathology and offering me an interesting look into the medical world. Working with you all made me see how technology and AI can directly impact clinical practice, and affirmed for me the importance of the medical applications of my work. I would also like to thank Sharvaree Vadgama for our weekly paper meetings at the beginning of this project, which greatly helped me in learning the technical preliminaries necessary to conduct this study. Moreover, I of course want to thank my family for supporting me unconditionally. And finally, I thank my friends from the former master AI room, who were always there struggling with me during this period and who made working on this thesis an almost enjoyable experience.

Reparameterization Gradients Through Acceptance-Rejection Sampling Algorithms

Abstract: Variational inference using the reparameterization trick has enabled large-scale approximate Bayesian inference in complex probabilistic models, leveraging stochastic optimization to sidestep intractable expectations. The reparameterization trick is applicable when we can simulate a random variable by applying a differentiable deterministic function on an auxiliary random variable whose distribution is fixed. For many distributions of interest (such as the gamma or Dirichlet), simulation of random variables relies on acceptance-rejection sampling. The discontinuity introduced by the accept-reject step means that standard reparameterization tricks are not applicable. We propose a new method that lets us leverage reparameterization gradients even when variables are outputs of a acceptance-rejection sampling algorithm. Our approach enables reparameterization on a larger class of variational distributions. In several studies of real and synthetic data, we show that the variance of the estimator of the gradient is significantly lower than other state-of-the-art methods. This leads to faster convergence of stochastic gradient variational inference.

reparameterization trick
acceptance-rejection sampling schemes


Why Should We Care About Threshold Networks?

Specifically, these networks have significantly lower memory footprint, less computational complexity, and consume less energy #REFR .

Neural networks with threshold activations are highly desirable due to the following reasons:

• Since the threshold activation (1) is restricted to take values in {0, s}, threshold neural network models are far more suitable for hardware implementations #OTHEREFR .

• Modern neural networks have extremely large number of full precision trainable parameters so that several computational barriers emerge during hardware implementations. One approach to 7with the non-convex training heuristic STE.

We also indicate the time taken to solve the convex programs with markers.

For non-convex STE, we repeat the training with 5 different initializations.

In each case, our convex training algorithms achieve lower objective than all the non-convex heuristics (see Appendix B.5 for details).

mitigate these issues is reducing the network size by grouping the parameters via a hash function #OTHEREFR .

Globally Optimal Training Of Neural Net-Works With Threshold Activation Functions

Abstract: Threshold activation functions are highly preferable in neural networks due to their efficiency in hardware implementations. Moreover, their mode of operation is more interpretable and resembles that of biological neurons. However, traditional gradient based algorithms such as Gradient Descent cannot be used to train the parameters of neural networks with threshold activations since the activation function has zero gradient except at a single non-differentiable point. To this end, we study weight decay regularized training problems of deep neural networks with threshold activations. We first show that regularized deep threshold network training problems can be equivalently formulated as a standard convex optimization problem, which parallels the LASSO method, provided that the last hidden layer width exceeds a certain threshold. We also derive a simplified convex optimization formulation when the dataset can be shattered at a certain layer of the network. We corroborate our theoretical results with various numerical experiments.

Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization

Abstract: Optimization of Binarized Neural Networks (BNNs) currently relies on real-valued latent weights to accumulate small update steps. In this paper, we argue that these latent weights cannot be treated analogously to weights in real-valued networks. Instead their main role is to provide inertia during training. We interpret current methods in terms of inertia and provide novel insights into the optimization of BNNs. We subsequently introduce the first optimizer specifically designed for BNNs, Binary Optimizer (Bop), and demonstrate its performance on CIFAR-10 and ImageNet. Together, the redefinition of latent weights as inertia and the introduction of Bop enable a better understanding of BNN optimization and open up the way for further improvements in training methodologies for BNNs.


We also include Hit@10 as an additional metric following knowledge graph embedding literature and GraIL #REFR .

SpaLoc also has comparable performance with KE methods on the FB15K-237 datasets, outperforming GraIL by a large margin.

Comparing SpaLoc with node embedding-based methods (TransE) and GNN-based methods (GraIL) that only consider binary edges, we see that our hyperedge-based model enables better relation prediction that requires reasoning about other entities.

The necessity of hyperedges is further supported by Appendix E, where we show that setting the maximum arity of SpaLoc to 2 (i.e., removing hyperedges) significantly degrades the performance.

Notably, in contrast to other methods for the transductive setting that store entity embeddings for all knowledge graph nodes, SpaLoc directly uses the inductive learning setting.

That is, SpaLoc does Table 5 : Results of transductive link prediction on real-world knowledge graphs.

The scores and standard errors (appendix Table 8 not store entity information about each knowledge graph node.

SpaLoc performs pretty well in the classification setting and ranking setting when negative candidate sets are small (Table 5 ).

However, we find inductive reasoning methods, including SpaLoc, GraIL #OTHEREFR , and TACT #OTHEREFR , perform poorly in the traditional knowledge graph ranking setting #OTHEREFR (i.e., ranking the positive triplets against negative triplets of all replacement of head/tail) entities.

The ranking scores are close to zero when the negative candidates set are large.

We attribute this phenomenon to the fact that the inductive models do not store entity information.

Sparse And Local Networks For Hypergraph Reasoning

Abstract: Reasoning about the relationships between entities from input facts (e.g., whether Ari is a grandparent of Charlie) generally requires explicit consideration of other entities that are not mentioned in the query (e.g., the parents of Charlie). In this paper, we present an approach for learning to solve problems of this kind in large, real-world domains, using sparse and local hypergraph neural networks (SpaLoc). SpaLoc is motivated by two observations from traditional logic-based reasoning: relational inferences usually apply locally (i.e., involve only a small number of individuals), and relations are usually sparse (i.e., only hold for a small percentage of tuples in a domain). We exploit these properties to make learning and inference efficient in very large domains by (1) using a sparse tensor representation for hypergraph neural networks, (2) applying a sparsification loss during training to encourage sparse representations, and (3) subsampling based on a novel information sufficiency-based sampling process during training. SpaLoc achieves state-of-the-art performance on several real-world, large-scale knowledge graph reasoning benchmarks, and is the first framework for applying hypergraph neural networks on real-world knowledge graphs with more than 10k nodes.

Modeling Relational Data With Graph Convolutional Networks

Abstract: Knowledge bases play a crucial role in many applications, for example question answering and information retrieval. Despite the great effort invested in creating and maintaining them, even the largest representatives (e.g., Yago, DBPedia or Wikidata) are highly incomplete. We introduce relational graph convolutional networks (R-GCNs) and apply them to two standard knowledge base completion tasks: link prediction (recovery of missing facts, i.e.~subject-predicate-object triples) and entity classification (recovery of missing attributes of entities). R-GCNs are a generalization of graph convolutional networks, a recent class of neural networks operating on graphs, and are developed specifically to deal with highly multi-relational data, characteristic of realistic knowledge bases. Our methods achieve competitive performance on standard benchmarks for both tasks, demonstrating especially promising results on the challenging FB15k-237 subset of Freebase.


The experimental results on the standard MSCOCO image captioning dataset #REFR show an improved or comparable performance comparing with several state-of-the-art methods in fair conditions.

This allows the captioning system to exploit both spatial and semantic features simultaneously, and makes end-to-end training of the system possible.

Finally, a multi-modal reward function is proposed for DRL in a two-phase supervised-reinforced training process of the captioning system.

This reward function consists of CIDEr, as the language reward, and an image-caption similarity as vision reward.

The language reward measures the similarity between the generated and ground truth captions.

The vision reward measures the similarity between the generated caption and the image contents, obtained from an embedding network for mapping both caption and image into a common latent space.

Detailed experimental analysis reveal the effectiveness of the visual relationships information on improving the performance of the captioning system based on common evaluation metrics.

Moreover, the results indicate that using the proposed multi-modal reward for DRL with SCST algorithm outperforms the original version of this algorithm.

In summary, the main contributions of this study are as follows:

• Scene graph-based visual relationship triplets are introduced as high-level semantic features to improve the image captioning results.

• An AoA-based deep neural architecture is proposed for fusion of visual relationships and spatial features extracted from images in the decoder of the captioning system.

Multi-Modal Reward For Visual Relationships-Based Image Captioning

Abstract: Deep neural networks have achieved promising results in automatic image captioning due to their effective representation learning and context-based content generation capabilities. As a prominent type of deep features used in many of the recent image captioning methods, the well-known bottomup features provide a detailed representation of different objects of the image in comparison with the feature maps directly extracted from the raw image. However, the lack of high-level semantic information about the relationships between these objects is an important drawback of bottom-up features, despite their expensive and resource-demanding extraction procedure. To take advantage of visual relationships in caption generation, this paper proposes a deep neural network architecture for image captioning based on fusing the visual relationships information extracted from an image's scene graph with the spatial feature maps of the image. A multi-modal reward function is then introduced for deep reinforcement learning of the proposed network using a combination of language and vision similarities in a common embedding space. The results of extensive experimentation on the MSCOCO dataset show the effectiveness of using visual relationships in the proposed captioning method. Moreover, the results clearly indicate that the proposed multi-modal reward in deep reinforcement learning leads to better model optimization, outperforming several state-of-the-art image captioning algorithms, while using light and easy to extract image features. A detailed experimental study of the components constituting the proposed method is also presented.

Microsoft Coco Captions: Data Collection And Evaluation Server

Abstract: In this paper we describe the Microsoft COCO Caption dataset and evaluation server. When completed, the dataset will contain over one and a half million captions describing over 330,000 images. For the training and validation images, five independent human generated captions will be provided. To ensure consistency in evaluation of automatic caption generation algorithms, an evaluation server is used. The evaluation server receives candidate captions and scores them using several popular metrics, including BLEU, METEOR, ROUGE and CIDEr. Instructions for using the evaluation server are provided.


We follow #REFR and use the mean CE (mCE) as the primary metric in comparing models' robustness.

To normalize the severity effects, we choose Cen-terPoint #OTHEREFR and MinkUNet #OTHEREFR as the baseline models for the 3D detectors and segmentors, respectively. The CE and mCE scores are calculated as follows:

where Acc i,l denotes task-specific accuracy scores, i.e., mIoU, AP, NDS, or APH(L2), on corruption type i at severity level l.

N = 8 is the total number of corruption types. Resilience Rate (RR).

We define mean RR (mRR) as the relative robustness indicator for measuring how much accuracy can a model retain when evaluated on the corruption sets. The RR and mRR scores are calculated as follows.

Robo3D: Towards Robust And Reliable 3D Perception Against Corruptions

Abstract: Figure 1: Taxonomy of the Robo3D benchmark. We simulate eight corruption types from three categories: 1) Severe weather conditions, such as fog, rain, and snow; 2) External disturbances that are caused by motion blur or result in the missing of LiDAR beams; and 3) Internal sensor failure, including LiDAR crosstalk, possible incomplete echo, and cross-sensor scenarios. Each corruption is further split into three levels (light, moderate, and heavy) based on its severity.

Benchmarking Neural Network Robustness To Common Corruptions And Surface Variations

Abstract: In this paper we establish rigorous benchmarks for image classifier robustness. Our first benchmark, ImageNet-C, standardizes and expands the corruption robustness topic, while showing which classifiers are preferable in safety-critical applications. Then we propose a new dataset called ImageNet-P which enables researchers to benchmark a classifier's robustness to common perturbations. Unlike recent robustness research, this benchmark evaluates performance on common corruptions and perturbations not worst-case adversarial perturbations. We find that there are negligible changes in relative corruption robustness from AlexNet classifiers to ResNet classifiers. Afterward we discover ways to enhance corruption and perturbation robustness. We even find that a bypassed adversarial defense provides substantial common perturbation robustness. Together our benchmarks may aid future work toward networks that robustly generalize.


The algorithm in #REFR produces accuracies that are comparable to BatchNorm, but unlike the case with BatchNorm, they found that introducing dropout layers improved performance further.

In the 2018 paper #OTHEREFR , population statistics were used instead of batch statistics, and a regularization term was included for the parameter.

In these experiments, it was noted that for batches of size larger than 32, the population statistics function as well as the batch statistics.

It was also noted that BatchNorm introduces Gaussian noise into the mean and variance parameters.

They also state that BatchNorm has very similar effects to norm regularization where the -norm is defined as

If norm regularization was sufficient, there would be no need for an optimization such as BatchNorm, since regularization is less computationally intensive.

However, this regularization claim is not consistent with other empirical studies, such as that presented in the 2018 paper #OTHEREFR .

An Empirical Analysis Of The Shift And Scale Parameters In Batchnorm

Abstract: Batch Normalization (BatchNorm) is a technique that improves the training of deep neural networks, especially Convolutional Neural Networks (CNN). It has been empirically demonstrated that BatchNorm increases performance, stability, and accuracy, although the reasons for such improvements are unclear. BatchNorm includes a normalization step as well as trainable shift and scale parameters. In this paper, we empirically examine the relative contribution to the success of BatchNorm of the normalization step, as compared to the re-parameterization via shifting and scaling. To conduct our experiments, we implement two new optimizers in PyTorch, namely, a version of BatchNorm that we refer to as AffineLayer, which includes the re-parameterization step without normalization, and a version with just the normalization step, that we call BatchNorm-minus. We compare the performance of our AffineLayer and BatchNorm-minus implementations to standard BatchNorm, and we also compare these to the case where no batch normalization is used. We experiment with four ResNet architectures (ResNet18, ResNet34, ResNet50, and ResNet101) over a standard image dataset and multiple batch sizes. Among other findings, we provide empirical evidence that the success of BatchNorm may derive primarily from improved weight initialization.

Towards Understanding Regularization In Batch Normalization

Abstract: Batch Normalization (BN) improves both convergence and generalization in training neural networks. This work understands these phenomena theoretically. We analyze BN by using a basic block of neural networks, consisting of a kernel layer, a BN layer, and a nonlinear activation function. This basic network helps us understand the impacts of BN in three aspects. First, by viewing BN as an implicit regularizer, BN can be decomposed into population normalization (PN) and gamma decay as an explicit regularization. Second, learning dynamics of BN and the regularization show that training converged with large maximum and effective learning rate. Third, generalization of BN is explored by using statistical mechanics. Experiments demonstrate that BN in convolutional neural networks share the same traits of regularization as the above analyses.


Results are reported using ResNet-50 as backbone on the YTVIS 2019 #REFR benchmark. Video Mask: using YTVIS video mask labels.

To further enforce temporal consistency through the video clip, TK-Loss is employed in a cyclic manner instead of using dense frame-wise connections. This greatly reduces memory cost with negligible performance drop.

We extensively evaluate our MaskFreeVIS on four largescale VIS benchmarks, i.e., YouTube-VIS 2019/2021 #OTHEREFR , OVIS #OTHEREFR , and BDD100K MOTS #OTHEREFR .

MaskFreeVIS achieves competitive VIS performance without using any video masks or even image mask labels on all datasets.

Validated on various methods and backbones, MaskFreeVIS achieves 91.25% performance of its fully supervised counterparts, even outperforming a few recent fully-supervised methods #OTHEREFR on the popular YTVIS benchmark.

Our simple yet effective design greatly narrows the performance gap between weakly-supervised and fully- Table 1 . Mask annotation requirement for state-of-the-art VIS methods.

Image Mask: using COCO #OTHEREFR image mask labels for image-based pretraining.

Pseudo Video: using Pseudo Videos from COCO images for joint training #OTHEREFR . MaskFreeVIS achieves 91.5% (42.5 vs.

46.4) of its fully-supervised baseline performance (Mask2Former) without using any masks during training. supervised video instance segmentation.

It further demonstrates that expensive video masks, or even image masks, is not necessary for training high-performing VIS models.

Our contributions are summarized as follows: (i) To utilize temporal information, we develop a new parameterfree Temporal KNN-patch Loss, which leverages temporal masks consistency using unsupervised one-to-k patch correspondence. We extensively analyze the TK-Loss through ablative experiments.

Mask-Free Video Instance Segmentation

Abstract: Figure 1. Video instance segmentation (VIS) results of our MaskFreeVIS, trained without using any video or image mask annotation. By achieving a remarkable 42.5% mask AP on the YouTube-VIS val dataset, with a ResNet-50 backbone, our approach demonstrates that high-performing VIS can be learned even without any mask annotations.

Video Instance Segmentation

Abstract: In this paper we present a new computer vision task, named video instance segmentation. The goal of this new task is simultaneous detection, segmentation and tracking of instances in videos. In words, it is the first time that the image instance segmentation problem is extended to the video domain. To facilitate research on this new task, we propose a large-scale benchmark called YouTube-VIS, which consists of 2883 high-resolution YouTube videos, a 40-category label set and 131k high-quality instance masks. In addition, we propose a novel algorithm called MaskTrack R-CNN for this task. Our new method introduces a new tracking branch to Mask R-CNN to jointly perform the detection, segmentation and tracking tasks simultaneously. Finally, we evaluate the proposed method and several strong baselines on our new dataset. Experimental results clearly demonstrate the advantages of the proposed algorithm and reveal insight for future improvement. We believe the video instance segmentation task will motivate the community along the line of research for video understanding.


Worse yet, the protocol introduces an additional issue in that it requires a majority of miners to act faithfully and actually perform the (legally mandated) mutations-something that it cannot guarantee by design #REFR .

create and formally prove an editable blockchain protocol #OTHEREFR .

While any user can propose edits, the protocol ensures consensus-based voting on the proposals to prevent arbitrary edits.

This also means that no trusted third party is required.

The protocol is compatible with any consensus mechanism and even offers accountability of the performed edits.

#OTHEREFR While this solution removes the need for a trusted third party, it does not solve the other issue of redactable and forgetting blockchains: every individual node in the network still needs to be trusted, as mutations are published as chain updates as well.

In contrast to that, our solution functions even with of adversarial network participants, as noted above.

Furthermore, we do not depend on the honesty of the miners and, better still, do not require any changes of the blockchain software.

Decentralized Inverse Transparency With Blockchain

Abstract: Employee data can be used to facilitate work, but their misusage may pose risks for individuals. Inverse transparency therefore aims to track all usages of personal data, allowing individuals to monitor them to ensure accountability for potential misusage. This necessitates a trusted log to establish an agreed-upon and non-repudiable timeline of events. The unique properties of blockchain facilitate this by providing immutability and availability. For power asymmetric environments such as the workplace, permissionless blockchain is especially beneficial as no trusted third party is required. Yet, two issues remain: (1) In a decentralized environment, no arbiter can facilitate and attest to data exchanges. Simple peer-to-peer sharing of data, conversely, lacks the required non-repudiation. (2) With data governed by privacy legislation such as the GDPR, the core advantage of immutability becomes a liability. After a rightful request, an individual's personal data need to be rectified or deleted, which is impossible in an immutable blockchain. To solve these issues, we present Kovacs, a decentralized data exchange and usage logging system for inverse transparency built on blockchain. Its new-usage protocol ensures non-repudiation, and therefore accountability, for inverse transparency. Its one-time pseudonym generation algorithm guarantees unlinkability and enables proof of ownership, which allows data subjects to exercise their legal rights regarding their personal data. With our implementation, we show the viability of our solution. The decentralized communication impacts performance and scalability, but exchange duration and storage size are still reasonable. More importantly, the provided information security meets high requirements. We conclude that Kovacs realizes decentralized inverse transparency through secure and GDPR-compliant use of permissionless blockchain. CCS Concepts: • Computer systems organization → Peer-to-peer architectures; • Security and privacy → Distributed systems security; Privacy-preserving protocols; Cryptography.

Redactable Blockchain In The Permissionless Setting

Abstract: Bitcoin is an immutable permissionless blockchain system that has been extensively used as a public bulletin board by many different applications that heavily relies on its immutability. However, Bitcoin's immutability is not without its fair share of demerits. Interpol exposed the existence of harmful and potentially illegal documents, images and links in the Bitcoin blockchain, and since then there have been several qualitative and quantitative analysis on the types of data currently residing in the Bitcoin blockchain. Although there is a lot of attention on blockchains, surprisingly the previous solutions proposed for data redaction in the permissionless setting are far from feasible, and require additional trust assumptions. Hence, the problem of harmful data still poses a huge challenge for law enforcement agencies like Interpol (Tziakouris, IEEE SP if a redaction gathers enough votes the operation is performed on the chain. As an extra feature, our protocol offers public verifiability and accountability for the redacted chain. Moreover, we provide formal security definitions and proofs showing that our protocol is secure against redactions that were not agreed by consensus. Additionally, we show the viability of our approach with a proof-of-concept implementation that shows only a tiny overhead in the chain validation of our protocol when compared to an immutable one.