Recent research has shown that CNNs are often overly sensitive to high-frequency textural patterns.Inspired by the intuition that humans are more sensitive to the lower-frequency (larger-scale) patterns we design a regularization scheme that penalizes large differences between adjacent components within each convolutional kernel.We apply our regularization onto several popular training methods, demonstrating that the models with the proposed smooth kernels enjoy improved adversarial robustness.Further, building on recent work establishing connections between adversarial robustness and interpretability, we show that our method appears to give more perceptually-aligned gradients.

Introduciamo una regolarizzazione di smoothness per i kernel convoluzionali di CNN che può aiutare a migliorare la robustezza avversaria e portare a gradienti percettivamente allineati

Questo articolo propone un nuovo schema di regolarizzazione che incoraggia i kernel convoluzionali ad essere più smooth, sostenendo che ridurre la dipendenza della rete neurale dalle componenti ad alta frequenza aiuta la robustezza contro gli esempi adversarial.

Gli autori propongono un metodo per l'apprendimento di kernel convoluzionali più smooth, in particolare, un regolarizzatore che penalizza i grandi cambiamenti tra pixel consecutivi del kernel con l'intuizione di penalizzare l'uso di componenti di input ad alta frequenza.


Despite an ever growing literature on reinforcement learning algorithms and applications, much less is known about their statistical inference.In this paper, we investigate the large-sample behaviors of the Q-value estimates with closed-form characterizations of the asymptotic variances.This allows us to efficiently construct confidence regions for Q-value and optimal value functions, and to develop policies to minimize their estimation errors.This also leads to a policy exploration strategy that relies on estimating the relative discrepancies among the Q estimates.Numerical experiments show superior performances of our exploration strategy than other benchmark approaches.

Indaghiamo il comportamento su grandi campioni delle stime dei valori Q e abbiamo proposto una strategia di esplorazione efficiente che si basa sulla stima delle discrepanze relative tra le stime Q.


Entailment vectors are a principled way to encode in a vector what information is known and what is unknown.  They are designed to model relations where one vector should include all the information in another vector, called entailment.  This paper investigates the unsupervised learning of entailment vectors for the semantics of words.  Using simple entailment-based models of the semantics of words in text (distributional semantics), we induce entailment-vector word embeddings which outperform the best previous results for predicting entailment between words, in unsupervised and semi-supervised experiments on hyponymy.

Addestriamo word embeddings basati sull'entailment invece che sulla somiglianza, prevedendo con successo l'entailment lessicale.

L'articolo presenta un algoritmo di word embedding per l'entailment lessicale che segue il lavoro di Henderson e Popa (ACL, 2016).


We describe a simple scheme that allows an agent to learn about its environment in an unsupervised manner.Our scheme pits two versions of the same agent, Alice and Bob, against one another.Alice proposes a task for Bob to complete; and then Bob attempts to complete the task.  In this work we will focus on two kinds of environments: (nearly) reversible environments and environments that can be reset.Alice will "propose" the task by doing a sequence of actions and then Bob must undo or repeat them, respectively.  Via an appropriate reward structure, Alice and Bob automatically generate a curriculum of exploration, enabling unsupervised training of the agent.When Bob is deployed on an RL task within the environment, this unsupervised training reduces the number of supervised episodes needed to learn, and in some cases converges to a higher reward.

Unsupervised learning per reinforcement learning utilizzando un curriculum automatico di self-play

Una nuova formulazione per esplorare l'ambiente in modo unsupervised per aiutare un task specifico in seguito, dove un agente propone task sempre più difficili e l'agente che impara cerca di realizzarli.

Un modello di self-play in cui un agente impara a proporre task che sono facili per lui ma difficili per un avversario, creando un moving target di obiettivi di self-play e curriculum di apprendimento.


Many real-world data sets are represented as graphs, such as citation links, social media, and biological interaction.The volatile graph structure makes it non-trivial to employ convolutional neural networks (CNN's) for graph data processing.Recently, graph attention network (GAT) has proven a promising attempt by combining graph neural networks with attention mechanism, so as to achieve massage passing in graphs with arbitrary structures.However, the attention in GAT is computed mainly based on the similarity between the node content, while the structures of the graph remains largely unemployed (except in masking the attention out of one-hop neighbors).In this paper, we propose an `````````````````````````````"ADaptive Structural Fingerprint" (ADSF) model to fully exploit both topological details of the graph and  content features of the nodes.The key idea is to contextualize each node with a weighted, learnable receptive field  encoding rich and diverse local graph structures.By doing this, structural interactions between the nodes can  be inferred accurately, thus improving subsequent attention layer as well as the convergence of learning.Furthermore, our model provides a useful platform  for different subspaces of node features and various scales of graph structures to ``cross-talk'' with each other through the learning of multi-head attention, being particularly useful in handling complex real-world data.  Encouraging performance is observed on a number of benchmark data sets in node classification.

Sfruttare i ricchi dettagli strutturali nei dati strutturati a grafo tramite "impronte digitali strutturali" adattive

Una metodologia basata sulla struttura del grafo per arricchire il meccanismo di attention delle graph neural networks, con l'idea principale di esplorare le interazioni tra diversi tipi di nodi nelle vicinanze di un nodo radice.

Questo articolo estende l'idea di self-attention nelle NN a grafo, che è tipicamente basata sulla somiglianza delle caratteristiche tra i nodi, per includere la somiglianza strutturale.


Informed and robust decision making in the face of uncertainty is critical for robots that perform physical tasks alongside people.We formulate this as a Bayesian Reinforcement Learning problem over latent Markov Decision Processes (MDPs).While Bayes-optimality is theoretically the gold standard, existing algorithms do not scale well to continuous state and action spaces.We propose a scalable solution that builds on the following insight: in the absence of uncertainty, each latent MDP is easier to solve.We split the challenge into two simpler components.First, we obtain an ensemble of clairvoyant experts and fuse their advice to compute a baseline policy.Second, we train a Bayesian residual policy to improve upon the ensemble's recommendation and learn to reduce uncertainty.Our algorithm, Bayesian Residual Policy Optimization (BRPO), imports the scalability of policy gradient methods as well as the initialization from prior models.BRPO significantly improves the ensemble of experts and drastically outperforms existing adaptive RL methods.

Proponiamo un algoritmo scalabile di Bayesian Reinforcement Learning che impara una correzione bayesiana su un ensemble di esperti chiaroveggenti per risolvere problemi con reward e dinamiche latenti complesse.

Questo articolo considera il problema del reifnorcement learning bayesiano su processi decisionali di Markov latenti (MDP) prendendo decisioni con esperti.

In questo articolo, gli autori motivano e propongono un algoritmo di apprendimento, chiamato Bayesian Residual Policy Optimization (BRPO), per problemi di reinforcement learning bayesiani.


One of the mysteries in the success of neural networks is randomly initialized first order methods like gradient descent can achieve zero training loss even though the objective function is non-convex and non-smooth.This paper demystifies this surprising phenomenon for two-layer fully connected ReLU activated neural networks.For an $m$ hidden node shallow neural network with ReLU activation and $n$ training data, we show as long as $m$ is large enough and no two inputs are parallel, randomly initialized gradient descent converges to a globally optimal solution at a linear convergence rate for the quadratic loss function.Our analysis relies on the following observation: over-parameterization and random initialization jointly restrict every weight vector to be close to its initialization for all iterations, which allows us to exploit a strong convexity-like property to show that gradient descent converges at a global linear rate to the global optimum.We believe these insights are also useful in analyzing deep models and other first order methods.

Dimostriamo che gradient descent raggiunge un valore di zero nella loss di training con un rate lineare su reti neurali sovra-parametrizzate.

Questo lavoro considera l'ottimizzazione di una rete ReLU a due layer sovraparametrizzata con la loss quadratica e con un dataset con etichette arbitrarie.

Questo articolo studia le reti neurali con un hidden layer con loss quadratica, dove dimostrano che nell'impostazione sovra-parametrizzata, l'inizializzazione casuale e la gradient descent arriva a loss zero.


For many applications, in particular in natural science, the task is todetermine hidden system parameters from a set of measurements.Often,the forward process from parameter- to measurement-space is well-defined,whereas the inverse problem is ambiguous: multiple parameter sets canresult in the same measurement.To fully characterize this ambiguity, the fullposterior parameter distribution, conditioned on an observed measurement,has to be determined.We argue that a particular class of neural networksis well suited for this task â€“ so-called Invertible Neural Networks (INNs).Unlike classical neural networks, which attempt to solve the ambiguousinverse problem directly, INNs focus on learning the forward process, usingadditional latent output variables to capture the information otherwiselost.Due to invertibility, a model of the corresponding inverse process islearned implicitly.Given a specific measurement and the distribution ofthe latent variables, the inverse pass of the INN provides the full posteriorover parameter space.We prove theoretically and verify experimentally, onartificial data and real-world problems from medicine and astrophysics, thatINNs are a powerful analysis tool to find multi-modalities in parameter space,uncover parameter correlations, and identify unrecoverable parameters.

Analizzare problemi inversi con reti neurali invertibili

L'autore propone di utilizzare le reti invertibili per risolvere problemi inversi ambigui e suggerisce di non addestrare solo il modello forward, ma anche il modello inverso con un critico MMD.

Il paper propone una rete invertibile con osservazioni per la probabilità posteriore di distribuzioni di input complesse con uno schema di training bidirezionale teoricamente valido.


Decisions made by machine learning systems have increasing influence on the world.Yet it is common for machine learning algorithms to assume that no such influence exists.An example is the use of the i.i.d.assumption in online learning for applications such as content recommendation, where the (choice of) content displayed can change users' perceptions and preferences, or even drive them away, causing a shift in the distribution of users.Generally speaking, it is possible for an algorithm to change the distribution of its own inputs.We introduce the term self-induced distributional shift (SIDS) to describe this phenomenon.A large body of work in reinforcement learning and causal machine learning aims to deal with distributional shift caused by deploying learning systems previously trained offline.Our goal is similar, but distinct: we point out that changes to the learning algorithm, such as the introduction of meta-learning, can reveal hidden incentives for distributional shift (HIDS), and aim to diagnose and prevent problems associated with hidden incentives.We design a simple Â environment as a "unit test" for HIDS, as well as a content recommendation environment which allows us to disentangle different types of SIDS.Â  We demonstrate the potential for HIDS to cause unexpected or undesirable behavior in these environments, and propose and test a mitigation strategy.Â

Le metriche di performance sono specifiche incomplete; il fine non sempre giustifica i mezzi.

Gli autori mostrano come il meta-learning rivela gli incentivi nascosti per il distributional shift e propongono un approccio basato sullo scambio di learner tra gli ambienti per ridurre il distributional shift introdotto da loro stessi.

L'articolo generalizza l'incentivo intrinseco per il learner a vincere rendendo il task più facile nel meta-learning a una classe più ampia di problemi.


In one-class-learning tasks, only the normal case can be modeled with data, whereas the variation of all possible anomalies is too large to be described sufficiently by samples.Thus, due to the lack of representative data, the wide-spread discriminative approaches cannot cover such learning tasks, and rather generative models, which attempt to learn the input density of the normal cases, are used.However, generative models suffer from a large input dimensionality (as in images) and are typically inefficient learners.We propose to learn the data distribution more efficiently with a multi-hypotheses autoencoder.Moreover, the model is criticized by a discriminator, which prevents artificial data modes not supported by data, and which enforces diversity across hypotheses.This consistency-based anomaly detection (ConAD) framework allows the reliable identification of outof- distribution samples.For anomaly detection on CIFAR-10, it yields up to 3.9% points improvement over previously reported results.On a real anomaly detection task, the approach reduces the error of the baseline models from 6.8% to 1.5%.

Proponiamo un approccio di rilevamento delle anomalie che combina la modellazione della classe di primo piano tramite densità locali multiple con adversarial training.

L'articolo propone una tecnica per rendere i modelli generativi più robusti rendendoli coerenti con la densità locale.


Generative Adversarial Networks (GAN) can achieve promising performance on learning complex data distributions on different types of data.In this paper, we first show that a straightforward extension of an existing GAN algorithm is not applicable to point clouds, because the constraint required for discriminators is undefined for set data.We propose a two fold modification to a GAN algorithm to be able to generate point clouds (PC-GAN).First, we combine ideas from hierarchical Bayesian modeling and implicit generative models by learning a hierarchical and interpretable sampling process.A key component of our method is that we train a posterior inference network for the hidden variables.Second, PC-GAN defines a generic framework that can incorporate many existing GAN algorithms.We further propose a sandwiching objective, which results in a tighter Wasserstein distance estimate than the commonly used dual form in WGAN.We validate our claims on the ModelNet40 benchmark dataset and observe that PC- GAN trained by the sandwiching objective achieves better results on test data than existing methods.We also conduct studies on several tasks, including generalization on unseen point clouds, latent space interpolation, classification, and image to point clouds transformation, to demonstrate the versatility of the proposed PC-GAN algorithm.

Proponiamo una variante GAN che impara a generare nuvole di punti. Diversi studi sono stati esplorati, tra cui una stima più precisa della distanza di Wasserstein, la generazione condizionale, la generalizzazione a nuvole di punti non visti e l'image to point cloud.

Questo articolo propone di utilizzare GAN per generare nuvole di punti 3D e introduce un obiettivo sandwich, facendo la media tra il limite superiore e inferiore della distanza di Wasserstein tra le distribuzioni.

Questo articolo propone un nuovo modello generativo per dati non ordinati, con una particolare applicazione alle nuvole di punti, che include un metodo di inferenza e una nuova funzione obiettivo.


Existing attention mechanisms, are mostly item-based in that a model is trained to attend to individual items in a collection (the memory) where each item has a predefined, fixed granularity, e.g., a character or a word.Intuitively, an area in the memory consisting of multiple items can be worth attending to as a whole.We propose area attention: a way to attend to an area of the memory, where each area contains a group of items that are either spatially adjacent when the memory has a 2-dimensional structure, such as images, or temporally adjacent for 1-dimensional memory, such as natural language sentences.Importantly, the size of an area, i.e., the number of items in an area or the level of aggregation, is dynamically determined via learning, which can vary depending on the learned coherence of the adjacent items.By giving the model the option to attend to an area of items, instead of only individual items, a model can attend to information with varying granularity.Area attention can work along multi-head attention for attending to multiple areas in the memory.We evaluate area attention on two tasks: neural machine translation (both character and token-level) and image captioning, and improve upon strong (state-of-the-art) baselines in all the cases.These improvements are obtainable with a basic form of area attention that is parameter free.In addition to proposing the novel concept of area attention, we contribute an efficient way for computing it by leveraging the technique of summed area tables.

L'articolo presenta un nuovo approccio per i meccanismi di attention che può beneficiare una serie di task come la traduzione automatica e la didascalia delle immagini.

Questo articolo estende gli attuali modelli di attention dal livello di parola alla combinazione di parole adiacenti, applicando i modelli agli oggetti composti da parole adiacenti fuse.


We identify a phenomenon, which we refer to as *multi-model forgetting*, that occurs when sequentially training multiple deep networks with partially-shared parameters; the performance of previously-trained models degrades as one optimizes a subsequent one, due to the overwriting of shared parameters.To overcome this, we introduce a statistically-justified weight plasticity loss that regularizes the learning of a model's shared parameters according to their importance for the previous models, and demonstrate its effectiveness when training two models sequentially and for neural architecture search.Adding weight plasticity in neural architecture search preserves the best models to the end of the search and yields improved results in both natural language processing and computer vision tasks.

Identifichiamo un fenomeno, il lavaggio del cervello neurale, e introduciamo una loss di plasticità del peso statisticamente giustificata per superarlo.

Questo articolo discute il fenomeno del "lavaggio del cervello neurale", che si riferisce al fatto che le prestazioni di un modello sono influenzate da un altro modello che ne condivide i parametri.


Revealing latent structure in data is an active field of research, having introduced exciting technologies such as variational autoencoders and adversarial networks, and is essential to push machine learning towards unsupervised knowledge discovery.However, a major challenge is the lack of suitable benchmarks for an objective and quantitative evaluation of learned representations.To address this issue we introduce Morpho-MNIST, a framework that aims to answer: "to what extent has my model learned to represent specific factors of variation in the data?"We extend the popular MNIST dataset by adding a morphometric analysis enabling quantitative comparison of trained models, identification of the roles of latent variables, and characterisation of sample diversity.We further propose a set of quantifiable perturbations to assess the performance of unsupervised and supervised methods on challenging tasks such as outlier detection and domain adaptation.

Questo articolo introduce Morpho-MNIST, una collezione di metriche di forma e perturbazioni, in un passo verso la valutazione quantitativa del representation learning.

Questo articolo discute il problema della valutazione e della diagnosi delle rappresentazioni apprese utilizzando un modello generativo.

Gli autori presentano un insieme di criteri per categorizzare le cifre di MNIST e un insieme di perturbazioni interessanti per modificare il dataset MNIST.


Exploration in environments with sparse rewards is a key challenge for reinforcement learning.How do we design agents with generic inductive biases so that they can explore in a consistent manner instead of just using local exploration schemes like epsilon-greedy?We propose an unsupervised reinforcement learning agent which learns a discrete pixel grouping model that preserves spatial geometry of the sensors and implicitly of the environment as well.We use this representation to derive geometric intrinsic reward functions, like centroid coordinates and area, and learn policies to control each one of them with off-policy learning.These policies form a basis set of behaviors (options) which allows us explore in a consistent way and use them in a hierarchical reinforcement learning setup to solve for extrinsically defined rewards.We show that our approach can scale to a variety of domains with competitive performance, including navigation in 3D environments and Atari games with sparse rewards.

esplorazione strutturata nel deep reinforcement learning attraverso la scoperta e il controllo dell'astrazione visiva unsupervised

L'articolo introduce astrazioni visive che sono utilizzate per il reinforcement learning, dove un algoritmo impara a "controllare" ogni astrazione così come a selezionare le opzioni per raggiungere il task complessivo.


Combinatorial optimization is a common theme in computer science.While in general such problems are NP-Hard, from a practical point of view, locally optimal solutions can be useful.In some combinatorial problems however, it can be hard to define meaningful solution neighborhoods that connect large portions of the search space, thus hindering methods that search this space directly.We suggest to circumvent such cases by utilizing a policy gradient algorithm that transforms the problem to the continuous domain, and to optimize a new surrogate objective that renders the former as generic stochastic optimizer.This is achieved by producing a surrogate objective whose distribution is fixed and predetermined, thus removing the need to fine-tune various hyper-parameters in a case by case manner.Since we are interested in methods which can successfully recover locally optimal solutions, we use the problem of finding locally maximal cliques as a challenging experimental benchmark, and we report results on a large dataset of graphs that is designed to test clique finding algorithms.Notably, we show in this benchmark that fixing the distribution of the surrogate is key to consistently recovering locally optimal solutions, and that our surrogate objective leads to an algorithm that outperforms other methods we have tested in a number of measures.

Un nuovo algoritmo policy gradient progettato per affrontare problemi di ottimizzazione combinatoria black-box. L'algoritmo si basa solo su valutazioni di funzioni e restituisce soluzioni localmente ottimali con alta probabilità.

L'articolo propone un approccio per costruire obiettivi surrogati per l'applicazione dei metodi policy gradient all'ottimizzazione combinatoria con lo scopo di ridurre la necessità di sintonizzazione degli iperparametri.

L'articolo propone di sostituire il termine di reward nell'algoritmo policy gradient con la sua distribuzione cumulativa empirica centrata.


Deterministic neural networks (NNs) are increasingly being deployed in safety critical domains, where calibrated, robust and efficient measures of uncertainty are crucial.While it is possible to train regression networks to output the parameters of a probability distribution by maximizing a Gaussian likelihood function, the resulting model remains oblivious to the underlying confidence of its predictions.In this paper, we propose a novel method for training deterministic NNs to not only estimate the desired target but also the associated evidence in support of that target.We accomplish this by  placing evidential priors over our original Gaussian likelihood function and training our NN to infer the hyperparameters of our evidential distribution.We impose priors during training such that the model is penalized when its predicted evidence is not aligned with the correct output.Thus the model estimates not only the probabilistic mean and variance of our target but also the underlying uncertainty associated with each of those parameters.We observe that our evidential regression method learns well-calibrated measures of uncertainty on various benchmarks, scales to complex computer vision tasks, and is robust to adversarial input perturbations.

Stima dell'incertezza veloce e calibrata per reti neurali senza campionamento

Questo articolo propone un nuovo approccio per stimare la fiducia delle predizioni in un ambiente di regressione, aprendo la porta ad applicazioni online con stime di incertezza completamente integrate.

Questo articolo ha proposto la regressione evidenziale deep, un metodo per il training delle reti neurali per stimare non solo l'output ma anche le prove associate a sostegno di quell'output.


The Lottery Ticket Hypothesis from Frankle & Carbin (2019) conjectures that, for typically-sized neural networks, it is possible to find small sub-networks which train faster and yield superior performance than their original counterparts.The proposed algorithm to search for such sub-networks (winning tickets), Iterative Magnitude Pruning (IMP), consistently finds sub-networks with 90-95% less parameters which indeed train faster and better than the overparameterized models they were extracted from, creating potential applications to problems such as transfer learning.In this paper, we propose a new algorithm to search for winning tickets, Continuous Sparsification, which continuously removes parameters from a network during training, and learns the sub-network's structure with gradient-based methods instead of relying on pruning strategies.We show empirically that our method is capable of finding tickets that outperforms the ones learned by Iterative Magnitude Pruning, and at the same time providing up to 5 times faster search, when measured in number of training epochs.

Proponiamo un nuovo algoritmo che trova rapidamente i winning ticket nelle reti neurali.

Questo articolo propone una nuova funzione obiettivo che può essere usata per ottimizzare congiuntamente un obiettivo di classificazione mentre incoraggia la sparsificazione in una rete che funziona con alta precisione.

Questo lavoro propone un nuovo metodo di pruning iterativo chiamato Continuous Sparsification, che riduce continuamente il peso corrente fino a raggiungere il rapporto target.


In most practical settings and theoretical analyses, one assumes that a model can be trained until convergence.However, the growing complexity of machine learning datasets and models may violate such assumptions.Indeed, current approaches for hyper-parameter tuning and neural architecture search tend to be limited by practical resource constraints.Therefore, we introduce a formal setting for studying training under the non-asymptotic, resource-constrained regime, i.e., budgeted training.We analyze the following problem: "given a dataset, algorithm, and fixed resource budget, what is the best achievable performance?"We focus on the number of optimization iterations as the representative resource.Under such a setting, we show that it is critical to adjust the learning rate schedule according to the given budget.Among budget-aware learning schedules, we find simple linear decay to be both robust and high-performing.We support our claim through extensive experiments with state-of-the-art models on ImageNet (image classification), Kinetics (video classification), MS COCO (object detection and instance segmentation), and Cityscapes (semantic segmentation).We also analyze our results and find that the key to a good schedule is budgeted convergence, a phenomenon whereby the gradient vanishes at the end of each allowed budget.We also revisit existing approaches for fast convergence and show that budget-aware learning schedules readily outperform such approaches under (the practical but under-explored) budgeted training setting.

Introdurre un'impostazione formale per un training con budget e proporre uno schedule lineare del learning rate consapevole del budget

Questo lavoro presenta una tecnica per adattare il learning rate per il training delle reti neurali sotto un numero fisso di epoche.

Questo articolo ha analizzato quale programma di learning rate dovrebbe essere usato quando il numero di iterazioni è limitato usando un nuovo concetto di BAS (Budget-Aware Schedule).


We present a new approach for efficient exploration which leverages a low-dimensional encoding of the environment learned with a combination of model-based and model-free objectives.Our approach uses intrinsic rewards that are based on a weighted distance of nearest neighbors in the low dimensional representational space to gauge novelty.We then leverage these intrinsic rewards for sample-efficient exploration with planning routines in representational space.One key element of our approach is that we perform more gradient steps in-between every environment step in order to ensure the model accuracy.We test our approach on a number of maze tasks, as well as a control problem and show that our exploration approach is more sample-efficient compared to strong baselines.

Conduciamo l'esplorazione usando ricompense intrinseche che si basano su una distanza ponderata dei nearest neighbor nello spazio delle rappresentazioni.

Questo articolo propone un metodo per un'esplorazione efficiente in MDP tabulari e anche un semplice ambiente di controllo, usando encoder deterministici per imparare una rappresentazione a bassa dimensione della dinamica dell'ambiente.

Questo articolo propone un metodo di esplorazione efficiente per agenti RL usando una combinazione di approcci basati sul modello e senza modello con una metrica di novità.


Neural networks are vulnerable to small adversarial perturbations.While existing literature largely focused on the vulnerability of learned models, we demonstrate an intriguing phenomenon that adversarial robustness, unlike clean accuracy, is sensitive to the input data distribution.Even a semantics-preserving transformations on the input data distribution can cause a significantly different robustness for the adversarially trained model that is both trained and evaluated on the new distribution.We show this by constructing semantically- identical variants for MNIST and CIFAR10 respectively, and show that standardly trained models achieve similar clean accuracies on them, but adversarially trained models achieve significantly different robustness accuracies.This counter-intuitive phenomenon indicates that input data distribution alone can affect the adversarial robustness of trained neural networks, not necessarily the tasks themselves.Lastly, we discuss the practical implications on evaluating adversarial robustness, and make initial attempts to understand this complex phenomenon.

Le prestazioni di robustezza dei modelli PGD trained sono sensibili alle trasformazioni semantics-preserving dei dataset di immagini, il che implica la difficoltà della valutazione degli algoritmi di reinforcement learning nella pratica.


Sample inefficiency is a long-lasting problem in reinforcement learning (RL).  The state-of-the-art uses action value function to derive policy while it usually involves an extensive search over the state-action space and unstable optimization.Towards the sample-efficient RL, we propose ranking policy gradient (RPG), a policy gradient method that learns the optimal rank of a set of discrete actions.  To accelerate the learning of policy gradient methods, we establish the equivalence between maximizing the lower bound of return and imitating a near-optimal policy without accessing any oracles.These results lead to a general off-policy learning framework, which preserves the optimality, reduces variance, and improves the sample-efficiency.We conduct extensive experiments showing that when consolidating with the off-policy learning framework, RPG substantially reduces the sample complexity, comparing to the state-of-the-art.

Proponiamo il ranking policy gradient che impara il rango ottimale delle azioni per massimizzare il rendimento. Proponiamo un framework generale di apprendimento off-policy con le proprietà di conservazione dell'ottimalità, riduzione della varianza e sample-efficiency.

Questo articolo propone di riparametrizzare la politica utilizzando una forma di ranking per convertire il problema RL in un problema di supervised learning.

Questo articolo presenta un nuovo punto di vista sui metodi policy gradient dal punto di vista del ranking.


We introduce MultiGrain, a neural network architecture that generates compact image embedding vectors that solve multiple tasks of different granularity: class, instance, and copy recognition.MultiGrain is trained jointly for classification by optimizing the cross-entropy loss and for instance/copy recognition by optimizing a self-supervised ranking loss.The self-supervised loss only uses data augmentation and thus does not require additional labels.Remarkably, the unified embeddings are not only much more compact than using several specialized embeddings, but they also have the same or better accuracy.When fed to a linear classifier, MultiGrain using ResNet-50 achieves 79.4% top-1 accuracy on ImageNet, a +1.8% absolute improvement over the the current state-of-the-art AutoAugment method.The same embeddings perform on par with state-of-the-art instance retrieval with images of moderate resolution.An ablation study shows that our approach benefits from the self-supervision, the pooling method and the mini-batches with repeated augmentations of the same image.

Combinando la classificazione e il recupero delle immagini in un'architettura di rete neurale, otteniamo un miglioramento per entrambi i task.

Questo articolo propone un embedding unificato per la classificazione delle immagini e il recupero delle istanze per migliorare le prestazioni di entrambi i task.

L'articolo propone di addestrare congiuntamente una deep neural network per la classificazione delle immagini, l'istanza e il riconoscimento delle copie.


In this paper, we investigate mapping the hyponymy relation of wordnet to feature vectors.  We aim to model lexical knowledge in such a way that it can be used as  input in generic machine-learning models, such as phrase entailment  predictors.  We propose two models.The first one leverages an existing mapping of  words to feature vectors (fasttext), and attempts to classify  such vectors as within or outside of each class.The second model is fully supervised,  using solely wordnet as a ground truth.It maps each concept to an  interval or a disjunction thereof.  On the first model, we approach, but not quite attain state of the  art performance.The second model can achieve near-perfect accuracy.

Studiamo la mappatura della relazione di iponimia di wordnet ai vettori di feature

Questo articolo studia come l'iponimia tra le parole può essere mappata nelle rappresentazioni delle feature.

Questo articolo esplora la nozione di iponimia nelle rappresentazioni vettoriali di parole e descrive un metodo per organizzare le relazioni WordNet in una struttura ad albero per definire l'iponimia.


Recurrent Neural Networks (RNNs) are powerful autoregressive sequence models for learning prevalent patterns in natural language.   Yet language generated by RNNs often shows several degenerate characteristics that are uncommon in human language; while fluent, RNN language production can be overly generic, repetitive, and even self-contradictory.  We postulate that the objective function optimized by RNN language models, which amounts to the overall perplexity of a text, is not expressive enough to capture the abstract qualities of good generation such as Griceâ€™s Maxims.In this paper, we introduce a general learning framework that can construct a decoding objective better suited for generation.Starting with a generatively trained RNN language model, our framework learns to construct a substantially stronger generator by combining several discriminatively trained models that can collectively address the limitations of RNN generation.  Human evaluation demonstrates that text generated by the resulting generator is preferred over  that  of  baselines  by  a  large  margin  and  significantly  enhances  the  overall coherence, style, and information content of the generated text.

Costruiamo un generatore di linguaggio naturale più forte addestrando in modo discriminatorio funzioni di scoring che classificano le generazioni candidate rispetto a varie qualità di buona scrittura.

Questo articolo propone di riunire più bias induttivi che sperano di correggere le incongruenze nella decodifica delle sequenze e propone di ottimizzare per i parametri di una combinazione predefinita di vari sotto-obiettivi.

Questo documento combina il modello linguistico RNN con diversi modelli trained in modo discriminatorio per migliorare la generazione del linguaggio.

Questo articolo propone di migliorare la generazione di modelli linguistici RNN utilizzando obiettivi aumentati ispirati alle massime di comunicazione di Grice.


In recent years, the efficiency and even the feasibility of traditional load-balancing policies are challenged by the rapid growth of cloud infrastructure with increasing levels of server heterogeneity and increasing size of cloud services and applications.In such many software-load-balancers heterogeneous systems, traditional solutions, such as JSQ, incur an increasing communication overhead, whereas low-communication alternatives, such as JSQ(d) and the recently proposed JIQ scheme are either unstable or provide poor performance.We argue that a better low-communication load balancing scheme can be established by allowing each dispatcher to have a different view of the system and keep using JSQ, rather than greedily trying to avoid starvation on a per-decision basis. accordingly, we introduce the Loosely-Shortest -Queue family of load balancing algorithms.Roughly speaking, in Loosely-shortest -Queue, each dispatcher keeps a different approximation of the server queue lengths and routes jobs to the shortest among them.Communication is used only to update the approximations and make sure that they are not too far from the real queue lengths in expectation.We formally establish the strong stability of any Loosely-Shortest -Queue policy and provide an easy-to-verify sufficient condition for verifying that a policy is Loosely-Shortest -Queue.We further demonstrate that the Loosely-Shortest -Queue approach allows constructing throughput optimal policies with an arbitrarily low communication budget.Finally, using extensive simulations that consider homogeneous, heterogeneous and highly skewed heterogeneous systems in scenarios with a single dispatcher as well as with multiple dispatchers, we show that the examined Loosely-Shortest -Queue example policies are always stable as dictated by theory.Moreover, it exhibits an appealing performance and significantly outperforms well-known low-communication policies, such as JSQ(d) and JIQ, while using a similar communication budget.

Soluzione di load balancing scalabile e a bassa comunicazione per sistemi multi-dispatcher a server eterogenei con forti garanzie teoriche e promettenti risultati empirici.


We propose a novel quantitative measure to predict the performance of a deep neural network classifier, where the measure is derived exclusively from the graph structure of the network.We expect that this measure is a fundamental first step in developing a method to evaluate new network architectures and reduce the reliance on the computationally expensive trial and error or "brute force" optimisation processes involved in model selection.The measure is derived in the context of multi-layer perceptrons (MLPs), but the definitions are shown to be useful also in the context of deep convolutional neural networks (CNN), where it is able to estimate and compare the relative performance of different types of neural networks, such as VGG, ResNet, and DenseNet.Our measure is also used to study the effects of some important "hidden" hyper-parameters of the DenseNet architecture, such as number of layers, growth rate and the dimension of 1x1 convolutions in DenseNet-BC.Ultimately, our measure facilitates the optimisation of the DenseNet design, which shows improved results compared to the baseline.

Una misura quantitativa per prevedere le prestazioni dei modelli di deep neural network.

L'articolo propone una nuova quantità che conta il numero di percorsi nella rete neurale che è predittiva delle prestazioni delle reti neurali con lo stesso numero di parametri.

L'articolo presenta un metodo per contare i percorsi nelle reti neurali deep che plausibilmente può essere utilizzato per misurare le prestazioni della rete.


There is a stark disparity between the learning rate schedules used in the practice of large scale machine learning and what are considered admissible learning rate schedules prescribed in the theory of stochastic approximation.Recent results, such as in the 'super-convergence' methods which use oscillating learning rates, serve to emphasize this point even more.One plausible explanation is that non-convex neural network training procedures are better suited to the use of fundamentally different learning rate  schedules, such as the ``cut the learning rate every constant number of epochs'' method (which more closely resembles an exponentially decaying learning rate schedule); note that this widely used schedule is in stark contrast to the polynomial decay schemes prescribed in the stochastic approximation literature, which are indeed shown to be (worst case) optimal for classes of convex optimization problems.The main contribution of this work shows that the picture is far more nuanced, where we do not even need to move to non-convex optimization to show other learning rate schemes can be far more effective.In fact, even for the simple case of stochastic linear regression with a fixed time horizon, the rate achieved by any polynomial decay scheme is sub-optimal compared to the statistical minimax rate (by a factor of condition number); in contrast the ```''cut the learning rate every constant number of epochs'' provides an exponential improvement (depending only logarithmically on the condition number) compared to any polynomial decay scheme.  Finally, it is important to ask if our theoretical insights are somehow fundamentally tied to quadratic loss minimization (where we have circumvented minimax lower bounds for more general convex optimization problems)?Here, we conjecture that recent results which make the gradient norm small at a near optimal rate, for both convex and non-convex optimization, may also provide more insights into learning rate schedules used in practice.

Questo articolo presenta uno studio rigoroso del perché le learning rate schedule usati praticamente (per un dato budget computazionale) offrono vantaggi significativi anche se questi schemi non sono sostenuti dalla teoria classica dell'approssimazione stocastica.

Questo articolo presenta uno studio teorico di diversi learning rate schedule che ha portato a limiti minimi statistici minimax per entrambi gli schemi polinomiali e constant-and-cut.

L'articolo studia l'effetto delle scelte di learning-rate per l'ottimizzazione stocastica, concentrandosi sul least-mean-squares con stepsize decrescenti


We present Value Propagation (VProp), a set of parameter-efficient differentiable planning modules built on Value Iteration which can successfully be trained using reinforcement learning to solve unseen tasks, has the capability to generalize to larger map sizes, and can learn to navigate in dynamic environments.We show that the modules enable learning to plan when the environment also includes stochastic elements, providing a cost-efficient learning system to build low-level size-invariant planners for a variety of interactive navigation problems.We evaluate on static and dynamic configurations of MazeBase grid-worlds, with randomly generated environments of several different sizes, and on a StarCraft navigation scenario, with more complex dynamics, and pixels as input.

Presentiamo pianificatori basati su convnet che sono sample-efficient e che si generalizzano a istanze più grandi di problemi di navigazione e pathfinding.

Propone metodi che possono essere visti come modifiche delle Value Iteration Network (VIN), con alcuni miglioramenti volti a migliorare la sample efficiency e la generalizzazione a grandi dimensioni dell'ambiente.

L'articolo presenta un'estensione delle value iteration network (VIN) considerando una funzione di transizione dipendente dallo stato.


Learning high-quality word embeddings is of significant importance in achieving better performance in many down-stream learning tasks.On one hand, traditional word embeddings are trained on a large scale corpus for general-purpose tasks, which are often sub-optimal for many domain-specific tasks.On the other hand, many domain-specific tasks do not have a large enough domain corpus to obtain high-quality embeddings.We observe that domains are not isolated and a small domain corpus can leverage the learned knowledge from many past domains to augment that corpus in order to generate high-quality embeddings.In this paper, we formulate the learning of word embeddings as a lifelong learning process.Given knowledge learned from many previous domains and a small new domain corpus, the proposed method can effectively generate new domain embeddings by leveraging a simple but effective algorithm and a meta-learner, where the meta-learner is able to provide word context similarity information at the domain-level.Experimental results demonstrate that the proposed method can effectively learn new domain embeddings from a small corpus and past domain knowledges\footnote{We will release the code after final revisions.}.Wealso demonstrate that general-purpose embeddings trained from a large scale corpus are sub-optimal in domain-specific tasks.

imparare migliori embedding di dominio attraverso lifelong learning e meta-learning

Presenta un metodo di lifelong learning per l'apprendimento di word embedding.

Questo articolo propone un approccio per imparare gli embedding in nuovi domini e batte significativamente la baseline su un task di aspect extraction.


Parameter pruning is a promising approach for CNN compression and acceleration by eliminating redundant model parameters with tolerable performance loss.Despite its effectiveness, existing regularization-based parameter pruning methods usually drive weights towards zero with large and constant regularization factors, which neglects the fact that the expressiveness of CNNs is fragile and needs a more gentle way of regularization for the networks to adapt during pruning.To solve this problem, we propose a new regularization-based pruning method (named IncReg) to incrementally assign different regularization factors to different weight groups based on their relative importance, whose effectiveness is proved on popular CNNs compared with state-of-the-art methods.

proponiamo un nuovo metodo di pruning basato sulla regolarizzazione (chiamato IncReg) per assegnare in modo incrementale diversi fattori di regolarizzazione a diversi gruppi di pesi in base alla loro importanza relativa.

Questo articolo propone un metodo di pruning basato sulla regolarizzazione per assegnare in modo incrementale diversi fattori di regolarizzazione a diversi gruppi di pesi in base alla loro importanza relativa.


Momentum based stochastic gradient methods such as heavy ball (HB) and Nesterov's accelerated gradient descent (NAG) method are widely used in practice for training deep networks and other supervised learning models, as they often provide significant improvements over stochastic gradient descent (SGD).Rigorously speaking, fast gradient methods have provable improvements over gradient descent only for the deterministic case, where the gradients are exact.In the stochastic case, the popular explanations for their wide applicability is that when these fast gradient methods are applied in the stochastic case, they partially mimic their exact gradient counterparts, resulting in some practical gain.This work provides a counterpoint to this belief by proving that there exist simple problem instances where these methods cannot outperform SGD despite the best setting of its parameters.These negative problem instances are, in an informal sense, generic; they do not look like carefully constructed pathological instances.These results suggest (along with empirical evidence) that HB or NAG's practical performance gains are a by-product of minibatching.Furthermore, this work provides a viable (and provable) alternative, which, on the same set of problem instances, significantly improves over HB, NAG, and SGD's performance.This algorithm, referred to as Accelerated Stochastic Gradient Descent (ASGD), is a simple to implement stochastic algorithm, based on a relatively less popular variant of Nesterov's Acceleration.Extensive empirical results in this paper show that ASGD has performance gains over HB, NAG, and SGD.The code for implementing the ASGD Algorithm can be found at https://github.com/rahulkidambi/AccSGD.

Gli schemi esistenti di momento/accelerazione come il metodo heavy ball e l'accelerazione di Nesterov impiegati con gradienti stocastici non migliorano rispetto al stochastic gradient descent standard, specialmente quando sono impiegati con batch size piccole.


Oversubscription planning (OSP) is the problem of finding plans that maximize the utility value of their end state while staying within a specified cost bound.Recently, it has been shown that OSP problems can be reformulated as classical planning problems with multiple cost functions but no utilities.  Here we take advantage of this reformulation to show that OSP problems can be solved optimally using the A* search algorithm, in contrast to previous approaches that have used variations on branch-and-bound search.This allows many powerful techniques developed for classical planning to be applied to OSP problems.We also introduce novel bound-sensitive heuristics, which are able to reason about the primary cost of a solution while taking into account secondary cost functions and bounds, to provide superior guidance compared to heuristics that do not take these bounds into account.We implement two such bound-sensitive variants of existing classical planning heuristics, and show experimentally that the resulting search is significantly more informed than comparable heuristics that do not consider bounds.

Mostriamo che i task di oversubscription planning possono essere risolti usando A* e introduciamo nuove euristiche sensibili ai bound per i task di oversubscription planning.

Presenta un approccio per risolvere in modo ottimale i task di oversubscription planning (OSP) utilizzando una traduzione alla pianificazione classica con funzioni di costo multiple.

L'articolo propone delle modifiche all'euristiche ammissibili per renderle meglio informate in un ambiente multi-criterio dove.


Previous work on adversarially robust neural networks requires large training sets and computationally expensive training procedures.  On the other hand, few-shot learning methods are highly vulnerable to adversarial examples.  The goal of our work is to produce networks which both perform well at few-shot tasks and are simultaneously robust to adversarial examples.  We adapt adversarial training for meta-learning, we adapt robust architectural features to small networks for meta-learning, we test pre-processing defenses as an alternative to adversarial training for meta-learning, and we investigate the advantages of robust meta-learning over robust transfer-learning for few-shot tasks.  This work provides a thorough analysis of adversarially robust methods in the context of meta-learning, and we lay the foundation for future work on defenses for few-shot tasks.

Sviluppiamo metodi di meta-learning per adversarially robust few-shot learning.

Questo articolo presenta un metodo che migliora la robustezza del few-shot learning introducendo l'adversarial attack di dati di query nella fase di fine-tuning dell'inner-task di un algoritmo di meta-learning.

Gli autori di questo articolo propongono un nuovo approccio per il training di un modello few-shot robusto.


Many of our core assumptions about how neural networks operate remain empirically untested.One common assumption is that convolutional neural networks need to be stable to small translations and deformations to solve image recognition tasks.For many years, this stability was baked into CNN architectures by incorporating interleaved pooling layers.Recently, however, interleaved pooling has largely been abandoned.This raises a number of questions: Are our intuitions about deformation stability right at all?Is it important?Is pooling necessary for deformation invariance?If not, how is deformation invariance achieved in its absence?In this work, we rigorously test these questions, and find that deformation stability in convolutional networks is more nuanced than it first appears: (1) Deformation invariance is not a binary property, but rather that different tasks require different degrees of deformation stability at different layers.(2) Deformation stability is not a fixed property of a network and is heavily adjusted over the course of training, largely through the smoothness of the convolutional filters.(3) Interleaved pooling layers are neither necessary nor sufficient for achieving the optimal form of deformation stability for natural image classification.(4) Pooling confers \emph{too much} deformation stability for image classification at initialization, and during training, networks have to learn to \emph{counteract} this inductive bias.Together, these findings provide new insights into the role of interleaved pooling and deformation invariance in CNNs, and demonstrate the importance of rigorous empirical testing of even our most basic assumptions about the working of neural networks.

Troviamo che il pooling da solo non determina la stabilità della deformazione nelle CNN e che la smoothness del filtro gioca un ruolo importante nel determinare la stabilità.


Deep neural networks (DNNs) have been shown to over-fit a dataset when being trained with noisy labels for a long enough time.To overcome this problem, we present a simple and effective method self-ensemble label filtering (SELF) to progressively filter out the wrong labels during training.Our method improves the task performance by gradually allowing supervision only from the potentially non-noisy (clean) labels and stops learning on the filtered noisy labels.For the filtering, we form running averages of predictions over the entire training dataset using the network output at different training epochs.We show that these ensemble estimates yield more accurate identification of inconsistent predictions throughout training than the single estimates of the network at the most recent training epoch.While filtered samples are removed entirely from the supervised training loss, we dynamically leverage them via semi-supervised learning in the unsupervised loss.We demonstrate the positive effect of such an approach on various image classification tasks under both symmetric and asymmetric label noise and at different noise ratios.It substantially outperforms all previous works on noise-aware learning across different datasets and can be applied to a broad set of network architectures.

Proponiamo un framework di self-ensemble per addestrare modelli di deep learning più robusti sotto dataset con label rumorose.

Questo articolo ha proposto il "self-ensemble label filtering" per l'apprendimento con label rumorose, dove il rumore della label è indipendente dall'istanza, che produce un'identificazione più accurata delle predizioni incoerenti.

Questo articolo propone un algoritmo per l'apprendimento da dati con label rumorose che alterna l'aggiornamento del modello alla rimozione dei campioni che sembrano avere label rumorose.


Long training times of deep neural networks are a bottleneck in machine learning research.The major impediment to fast training is the quadratic growth of both memory and compute requirements of dense and convolutional layers with respect to their information bandwidth.Recently, training `a priori' sparse networks has been proposed as a method for allowing layers to retain high information bandwidth, while keeping memory and compute low.However, the choice of which sparse topology should be used in these networks is unclear.In this work, we provide a theoretical foundation for the choice of intra-layer topology.First, we derive a new sparse neural network initialization scheme that allows us to explore the space of very deep sparse networks.Next, we evaluate several topologies and show that seemingly similar topologies can often have a large difference in attainable accuracy.To explain these differences, we develop a data-free heuristic that can evaluate a topology independently from the dataset the network will be trained on.We then derive a set of requirements that make a good topology, and arrive at a single topology that satisfies all of them.

Indaghiamo il pruning delle DNN prima del training e forniamo una risposta su quale topologia dovrebbe essere usata per il training di reti sparse a priori.

Gli autori propongono di sostituire i layer densi con layer lineari scarsamente connessi e un approccio per trovare la migliore topologia misurando quanto bene i layer scarsi approssimino i pesi casuali delle loro controparti dense.

L'articolo propone un'architettura sparsa a cascata che è una moltiplicazione di diverse matrici sparse e un modello di connettività specifico che supera altre considerazioni fornite.


Deep learning models require extensive architecture design exploration and hyperparameter optimization to perform well on a given task.The exploration of the model design space is often made by a human expert, and optimized using a combination of grid search and search heuristics over a large space of possible choices.Neural Architecture Search (NAS) is a Reinforcement Learning approach that has been proposed to automate architecture design.NAS has been successfully applied to generate Neural Networks that rival the best human-designed architectures.However, NAS requires sampling, constructing, and training hundreds to thousands of models to achieve well-performing architectures.This procedure needs to be executed from scratch for each new task.The application of NAS to a wide set of tasks currently lacks a way to transfer generalizable knowledge across tasks.In this paper, we present the Multitask Neural Model Search (MNMS) controller.Our goal is to learn a generalizable framework that can condition model construction on successful model searches for previously seen tasks, thus significantly speeding up the search for new tasks.We demonstrate that MNMS can conduct an automated architecture search for multiple tasks simultaneously while still learning well-performing, specialized models for each task.We then show that pre-trained MNMS controllers can transfer learning to new tasks.By leveraging knowledge from previous searches, we find that pre-trained MNMS models start from a better location in the search space and reduce search time on unseen tasks, while still discovering models that outperform published human-designed models.

Presentiamo Multitask Neural Model Search, un meta-learner che può progettare modelli per più task simultaneamente e trasferire l'apprendimento a task non visti prima.

Questo articolo estende la ricerca dell'architettura neurale al problema dell'apprendimento multitasking in cui un controllore di ricerca del modello condizionato dal task viene appreso per gestire più task contemporaneamente.

In questo articolo, gli autori riassumono il loro lavoro sulla costruzione di un framework, chiamato Multitask Neural Model Search controller, per la costruzione automatizzata di reti neurali attraverso più task contemporaneamente.


This work studies the problem of modeling non-linear visual processes by leveraging deep generative architectures for learning linear, Gaussian models of observed sequences.We propose a joint learning framework, combining a multivariate autoregressive model and deep convolutional generative networks.After justification of theoretical assumptions of inearization, we propose an architecture that allows Variational Autoencoders and Generative Adversarial Networks to simultaneously learn the non-linear observation as well as the linear state-transition model from a sequence of observed frames.Finally, we demonstrate our approach on conceptual toy examples and dynamic textures.

Modelliamo i processi visivi non lineari come rumore autoregressivo attraverso generative deep learning.

Propone un nuovo metodo che modella il processo visivo non lineare con una versione deep di un processo lineare (processo di Markov).

Questo articolo propone un nuovo modello generativo deep per sequenze, in particolare sequenze di immagini e video, che utilizza una struttura lineare in una parte del modello.


Partial differential equations (PDEs)  play a prominent role in many disciplines such as applied mathematics, physics, chemistry, material science, computer science, etc.PDEs are commonly derived based on physical laws or empirical observations.However, the governing equations for many complex systems in modern applications are still not fully known.With the rapid development of sensors, computational power, and data storage in the past decade, huge quantities of data can be easily collected and efficiently stored.Such vast quantity of data offers new opportunities for data-driven discovery of hidden physical laws.Inspired by the latest development of neural network designs in deep learning, we propose a new feed-forward deep network, called PDE-Net, to fulfill two objectives at the same time: to accurately predict dynamics of complex systems and to uncover the underlying hidden PDE models.The basic idea of the proposed PDE-Net is to learn differential operators by learning convolution kernels (filters), and apply neural networks or other machine learning methods to approximate the unknown nonlinear responses.Comparing with existing approaches, which either assume the form of the nonlinear response is known or fix certain finite difference approximations of differential operators, our approach has the most flexibility by learning both differential operators and the nonlinear responses.A special feature of the proposed PDE-Net is that all filters are properly constrained, which enables us to easily identify the governing PDE models while still maintaining the expressive and predictive power of the network.These constrains are carefully designed by fully exploiting the relation between the orders of differential operators and the orders of sum rules of filters (an important concept originated from wavelet theory).We also discuss relations of the PDE-Net with some existing networks in computer vision such as Network-In-Network (NIN) and Residual Neural Network (ResNet).Numerical experiments show that the PDE-Net has the potential to uncover the hidden PDE of the observed dynamics, and predict the dynamical behavior for a relatively long time, even in a noisy environment.

Questo articolo propone una nuova rete feed-forward, chiamata PDE-Net, per imparare le PDE dai dati.

L'articolo espone l'uso di architetture di deep learning allo scopo di identificare sistemi dinamici specificati da PDE.

L'articolo propone un algoritmo basato su reti neurali per l'apprendimento da dati che derivano da sistemi dinamici con equazioni che li goveranano che possono essere scritte come equazioni differenziali parziali.

Questo articolo affronta la modellazione di sistemi dinamici complessi attraverso equazioni differenziali parziali non parametriche utilizzando architetture neurali, dove l'idea più importante del paper (PDE-net) è quella di imparare sia gli operatori differenziali che la funzione che governa la PDE.


Each training step for a variational autoencoder (VAE) requires us to sample from the approximate posterior, so we usually choose simple (e.g. factorised) approximate posteriors in which sampling is an efficient computation that fully exploits GPU parallelism.  However, such simple approximate posteriors are often insufficient, as they eliminate statistical dependencies in the posterior.  While it is possible to use normalizing flow approximate posteriors for continuous latents, there is nothing analogous for discrete latents.The most natural approach to model discrete dependencies is an autoregressive distribution, but sampling from such distributions is inherently sequential and thus slow.  We develop a fast, parallel sampling procedure for autoregressive distributions based on fixed-point iterations which enables efficient and accurate variational inference in discrete state-space models.  To optimize the variational bound, we considered two ways to evaluate probabilities: inserting the relaxed samples directly into the pmf for the discrete distribution, or converting to continuous logistic latent variables and interpreting the K-step fixed-point iterations as a normalizing flow.  We found that converting to continuous latent variables gave considerable additional scope for mismatch between the true and approximate posteriors, which resulted in biased inferences, we thus used the former approach.  We tested our approach on the neuroscience problem of inferring discrete spiking activity from noisy calcium-imaging data, and found that it gave accurate connectivity estimates in an order of magnitude less time.

Presentiamo una procedura di sampling veloce simile al normalizing flow per modelli di variabili latenti discreti.

Questo articolo usa un'approssimazione variazionale di filtraggio autoregressivo per la stima dei parametri nei sistemi dinamici discreti usando iterazioni a virgola fissa.

Gli autori pongono una famiglia di probabilità posteriori autoregressiva generale per le variabili discrete o i loro rilassamenti continui.

Questo articolo ha due contributi principali: estende i normalizing flow alle impostazioni discrete e presenta una regola di aggiornamento approssimativa a virgola fissa per le serie temporali autoregressive che può sfruttare il parallelismo delle GPU.


Deep neural networks (DNNs) had great success on NLP tasks such as language modeling, machine translation and certain question answering (QA) tasks.However, the success is limited at more knowledge intensive tasks such as QA from a big corpus.Existing end-to-end deep QA models (Miller et al., 2016; Weston et al., 2014) need to read the entire text after observing the question, and therefore their complexity in responding a question is linear in the text size.This is prohibitive for practical tasks such as QA from Wikipedia, a novel, or the Web.We propose to solve this scalability issue by using symbolic meaning representations, which can be indexed and retrieved efficiently with complexity that is independent of the text size.More specifically, we use sequence-to-sequence models to encode knowledge symbolically and generate programs to answer questions from the encoded knowledge.We apply our approach, called the N-Gram Machine (NGM), to the bAbI tasks (Weston et al., 2015) and a special version of them (â€œlife-long bAbIâ€) which has stories of up to 10 million sentences.Our experiments show that NGM can successfully solve both of these tasks accurately and efficiently.Unlike fully differentiable memory models, NGMâ€™s time complexity and answering quality are not affected by the story length.The whole system of NGM is trained end-to-end with REINFORCE (Williams, 1992).To avoid high variance in gradient estimation, which is typical in discrete latent variable models, we use beam search instead of sampling.To tackle the exponentially large search space, we use a stabilized auto-encoding objective and a structure tweak procedure to iteratively reduce and refine the search space.

Proponiamo un framework che impara a codificare la conoscenza simbolicamente e a generare programmi per ragionare sulla conoscenza codificata.

Gli autori propongono la N-Gram machine per rispondere a domande su documenti lunghi.

Questo articolo presenta la n-gram machine, un modello che codifica le frasi in semplici rappresentazioni simboliche che possono essere interrogate in modo efficiente.


We propose to use a meta-learning objective that maximizes the speed of transfer on a modified distribution to learn how to modularize acquired knowledge.In particular, we focus on how to factor a joint distribution into appropriate conditionals, consistent with the causal directions.We explain when this can work, using the assumption that the changes in distributions are localized (e.g. to one of the marginals, for example due to an intervention on one of the variables).We prove that under this assumption of localized changes in causal mechanisms, the correct causal graph will tend to have only a few of its parameters with non-zero gradient, i.e. that need to be adapted (those of the modified variables).We argue and observe experimentally that this leads to faster adaptation, and use this property to define a meta-learning surrogate score which, in addition to a continuous parametrization of graphs, would favour correct causal graphs.Finally, motivated by the AI agent point of view (e.g. of a robot discovering its environment autonomously), we consider how the same objective can discover the causal variables themselves, as a transformation of observed low-level variables with no causal meaning.Experiments in the two-variable case validate the proposed ideas and theoretical results.

Questo articolo propone un obiettivo di meta-learning basato sulla velocità di adattamento alle distribuzioni di trasferimento per scoprire una decomposizione modulare e variabili causali.

L'articolo mostra che un modello con la struttura di base corretta si adatterà più velocemente a un intervento causale rispetto a un modello con una struttura non corretta.

In questo lavoro, gli autori hanno proposto un framework generale e sistematico dell'obiettivo di meta-trasfer che incorpora l'apprendimento della struttura causale sotto interventi sconosciuti.


Continual learning is a longstanding goal of artificial intelligence, but is often counfounded by catastrophic forgetting that prevents neural networks from learning tasks sequentially.Previous methods in continual learning have demonstrated how to mitigate catastrophic forgetting, and learn new tasks while retaining performance on the previous tasks.We analyze catastrophic forgetting from the perspective of change in classifier likelihood and propose a simple L1 minimization criterion which can be adapted to different use cases.We further investigate two ways to minimize forgetting as quantified by this criterion and propose strategies to achieve finer control over forgetting.Finally, we evaluate our strategies on 3 datasets of varying difficulty and demonstrate improvements over previously known L2 strategies for mitigating catastrophic forgetting.

Un'altra prospettiva sul catastrophic forgetting

Questo articolo introduce un framework per combattere il catastrophic forgetting basato sulla modifica del termine della loss per minimizzare i cambiamenti nella probabilità del classificatore, ottenuta tramite un'approssimazione in serie di Taylor.

Questo articolo cerca di risolvere il problema del continual learning concentrandosi sugli approcci di regolarizzazione, e propone una strategia L_1 per mitigare il problema.


We propose an approach to construct realistic 3D facial morphable models (3DMM) that allows an intuitive facial attributeediting workflow.Current face modeling methods using 3DMM suffer from the lack of local control.We thus create a 3DMM bycombining local part-based 3DMM for the eyes, nose, mouth, ears, and facial mask regions.Our local PCA-based approachuses a novel method to select the best eigenvectors from the local 3DMM to ensure that the combined 3DMM is expressivewhile allowing accurate reconstruction.The editing controls we provide to the user are intuitive as they are extracted fromanthropometric measurements found in the literature.Out of a large set of possible anthropometric measurements, we filter theones that have meaningful generative power given the face data set.We bind the measurements to the part-based 3DMM throughmapping matrices derived from our data set of facial scans.Our part-based 3DMM is compact yet accurate, and compared toother 3DMM methods, it provides a new trade-off between local and global control.We tested our approach on a data set of 135scans used to derive the 3DMM, plus 19 scans that served for validation.The results show that our part-based 3DMM approachhas excellent generative properties and allows intuitive local control to the user.

Proponiamo un approccio per costruire modelli realistici 3D morphable facial (3DMM) che permette un flusso di lavoro intuitivo di editing degli attributi facciali selezionando i migliori set di autovettori e misure antropometriche.

Propone un modello frammentario e morfabile per le mesh dei volti umani e propone anche una mappatura tra le misure antropometriche del volto e i parametri del modello al fine di sintetizzare e modificare i volti con gli attributi desiderati.

Questo articolo descrive un metodo di modello facciale morphable basato su parti che permette il controllo localizzato dell'utente.


We review eight machine learning classification algorithms to analyze Electroencephalographic (EEG) signals in order to distinguish EEG patterns associated with five basic educational tasks.There is a large variety of classifiers being used in this EEG-based Brain-Computer Interface (BCI) field.While previous EEG experiments used several classifiers in the same experiments or reviewed different algorithms on datasets from different experiments, our approach focuses on review eight classifier categories on the same dataset, including linear classifiers, non-linear Bayesian classifiers, nearest neighbour classifiers, ensemble methods, adaptive classifiers, tensor classifiers, transfer learning and deep learning.Besides, we intend to find an approach which can run smoothly on the current mainstream personal computers and smartphones.  The empirical evaluation demonstrated that Random Forest and LSTM (Long Short-Term Memory) outperform other approaches.We used a data set which users were conducting five frequently-conduct learning-related tasks, including reading, writing, and typing.Results showed that these best two algorithms could correctly classify different users with an accuracy increase of  5% to 9%, use each task independently.Within each subject, the tasks could be recognized with an accuracy increase of  4% to 7%, compared with other approaches.This work suggests that Random Forest could be a recommended approach (fast and accurate) for current mainstream hardware, while LSTM has the potential to be the first-choice approach when the mainstream computers and smartphones can process more data in a shorter time.

Due algoritmi hanno superato altri otto su un esperimento BCI basato su EEG


Multi-agent reinforcement learning offers a way to study how communication could emerge in communities of agents needing to solve specific problems.In this paper, we study the emergence of communication in the negotiation environment, a semi-cooperative model of agent interaction.We introduce two communication protocols - one grounded in the semantics of the game, and one which is a priori ungrounded.  We show that self-interested agents can use the pre-grounded communication channel to negotiate fairly, but are unable to effectively use the ungrounded, cheap talk channel to do the same.  However, prosocial agents do learn to use cheap talk to find an optimal negotiating strategy, suggesting that cooperation is necessary for language to emerge.We also study communication behaviour in a setting where one agent interacts with agents in a community with different levels of prosociality and show how agent identifiability can aid negotiation.

Insegniamo agli agenti a negoziare usando soloreinforcement learning; gli agenti egoisti possono farlo, ma solo usando un canale di comunicazione affidabile, e gli agenti prosociali possono negoziare usando il cheap talk.

Gli autori descrivono una variante del gioco di negoziazione con la considerazione di un canale di comunicazione secondario per il cheap talk, trovando che il canale secondario migliora i risultati di negoziazione.

Questo articolo esplora come gli agenti possono imparare a comunicare per risolvere un task di negoziazione e trovano che gli agenti prosociali sono in grado di imparare a basare i simboli usando RL, mentre gli agenti self-interested no.

Esamina i problemi di come gli agenti possono usare la comunicazione per massimizzare le loro reward in un semplice gioco di negoziazione.


The goal of few-shot learning is to learn a classifier that generalizes well even when trained with a limited number of training instances per class.The recently introduced meta-learning approaches tackle this problem by learning a generic classifier across a large number of multiclass classification tasks and generalizing the model to a new task.Yet, even with such meta-learning, the low-data problem in the novel classification task still remains.In this paper, we propose Transductive Propagation Network (TPN), a novel meta-learning framework for transductive inference that classifies the entire test set at once to alleviate the low-data problem.Specifically, we propose to learn to propagate labels from labeled instances to unlabeled test instances, by learning a graph construction module that exploits the manifold structure in the data.TPN jointly learns both the parameters of feature embedding and the graph construction in an end-to-end manner.  We validate TPN on multiple benchmark datasets, on which it largely outperforms existing few-shot learning approaches and achieves the state-of-the-art results.

Proponiamo un nuovo framework di meta-learning per l'inferenza trasduttiva che classifica l'intero test set in una sola volta per alleviare il problema dei pochi dati.

Questo articolo propone di affrontare few-shot learning in modo trasduttivo imparando un modello di propagazione delle label in una maniera end-to-end, è il primo paper dove si impara label propagation few-shot per l'apprendimento trasduttivo e ha prodotto utili risultati empirici.

Questo articolo propone un framework di meta-learning che sfrutta i dati non labelled imparando la propogazione delle label basata sul grafo in modo end-to-end.

Studia l'apprendimento few-shot in un'impostazione trasduttiva: usando il meta learning per imparare a propagare le label dai training sample ai test sample.


We describe the use of an automated scheduling system for observation policy design and to schedule operations of the NASA (National Aeronautics and Space Administration) ECOSystem Spaceborne Thermal Radiometer Experiment on Space Station (ECOSTRESS).We describe the adaptation of the Compressed Large-scale Activity Scheduler and Planner (CLASP) scheduling system to the ECOSTRESS scheduling problem, highlighting multiple use cases for automated scheduling and several challenges for the scheduling technology: handling long-term campaigns with changing information, Mass Storage Unit Ring Buffer operations challenges, and orbit uncertainty.The described scheduling system has been used for operations of the ECOSTRESS instrument since its nominal operations start July 2018 and is expected to operate until mission end in Summer 2019.

Descriviamo l'uso di un sistema di pianificazione automatizzato per la progettazione della policy di osservazione e per programmare le operazioni della missione ECOSTRESS della NASA.

Questo articolo presenta un adattamento di un sistema di programmazione automatica, CLASP, per indirizzare un esperimento EO (ECOSTRESS) sulla ISS.


Adversarial examples are modified samples that preserve original image structures but deviate classifiers.Researchers have put efforts into developing methods for generating adversarial examples and finding out origins.Past research put much attention on decision boundary changes caused by these methods.This paper, in contrast, discusses the origin of adversarial examples from a more underlying knowledge representation point of view.Human beings can learn and classify prototypes as well as transformations of objects.While neural networks store learned knowledge in a more hybrid way of combining all prototypes and transformations as a whole distribution.Hybrid storage may lead to lower distances between different classes so that small modifications can mislead the classifier.A one-step distribution imitation method is designed to imitate distribution of the nearest different class neighbor.Experiments show that simply by imitating distributions from a training set without any knowledge of the classifier can still lead to obvious impacts on classification results from deep networks.It also implies that adversarial examples can be in more forms than small perturbations.Potential ways of alleviating adversarial examples are discussed from the representation point of view.The first path is to change the encoding of data sent to the training step.Training data that are more prototypical can help seize more robust and accurate structural knowledge.The second path requires constructing learning frameworks with improved representations.

La memorizzazione e la rappresentazione hybrid della conoscenza appresa può essere una ragione per gli adversarial sample.


Differently from the popular Deep Q-Network (DQN) learning, Alternating Q-learning (AltQ) does not fully fit a target Q-function at each iteration, and is generally known to be unstable and inefficient.Limited applications of AltQ mostly rely on substantially altering the algorithm architecture in order to improve its performance.Although Adam appears to be a natural solution, its performance in AltQ has rarely been studied before.In this paper, we first provide a solid exploration on how well AltQ performs with Adam.We then take a further step to improve the implementation by adopting the technique of parameter restart.More specifically, the proposed algorithms are tested on a batch of Atari 2600 games and exhibit superior performance than the DQN learning method.The convergence rate of the slightly modified version of the proposed algorithms is characterized under the linear function approximation.To the best of our knowledge, this is the first theoretical study on the Adam-type algorithms in Q-learning.

Nuovi esperimenti e teoria per il Q-Learning basato su Adam

Questo articolo fornisce un risultato di convergenza per il tradizionale Q-learning con approssimazione di funzione lineare quando si usa un aggiornamento simile ad Adam.

Questo articolo descrive un metodo per migliorare l'algoritmo AltQ utilizzando una combinazione di un ottimizzatore Adam e riavviando regolarmente i parametri interni all'ottimizzatore Adam.


In search for more accurate predictive models, we customize capsule networks for the learning to diagnose problem.We also propose Spectral Capsule Networks, a novel variation of capsule networks, that converge faster than capsule network with EM routing.Spectral capsule networks  consist of spatial coincidence filters that detect entities based on the alignment of extracted features on a one-dimensional linear subspace.Experiments on a public benchmark learning to diagnose dataset not only shows the success of capsule networks on this task, but also confirm the faster convergence of the spectral capsule networks.

Una nuova capsule network che converge più velocemente sui nostri esperimenti di benchmark nell'ambito sanitario.

Presenta una variante delle capsule network che invece di usare il routing EM impiega un sottospazio lineare coperto dall'autovettore dominante sulla matrice dei voti pesati della capsule precedente.

L'articolo propone un metodo di routing migliorato, che impiega strumenti di decomposizione agli autovalori per trovare l'attivazione e la posa della capsule.


One of the big challenges in machine learning applications is that training data can be different from the real-world data faced by the algorithm.In language modeling, usersâ€™ language (e.g. in private messaging) could change in a year and be completely different from what we observe in publicly available data.At the same time, public data can be used for obtaining general knowledge (i.e. general model of English).We study approaches to distributed fine-tuning of a general model on user private data with the additional requirements of maintaining the quality on the general data and minimization of communication costs.We propose a novel technique that significantly improves prediction quality on usersâ€™ language compared to a general model and outperforms gradient compression methods in terms of communication efficiency.The proposed procedure is fast and leads to an almost 70% perplexity reduction and 8.7 percentage point improvement in keystroke saving rate on informal English texts.Finally, we propose an experimental framework for evaluating differential privacy of distributed training of language models and show that our approach has good privacy guarantees.

Proponiamo un metodo di fne-tuning distribuito dei language model sui dispositivi degli utenti senza raccolta di dati privati

Questo articolo si occupa di migliorare i language model su apparecchiature mobili basati su una piccola porzione di testo che l'utente ha immesso utilizzando un obiettivo interpolato linearmente tra il testo specifico dell'utente e l'inglese generale.


We propose that approximate Bayesian algorithms should optimize a new criterion, directly derived from the loss, to calculate their approximate posterior which we refer to as pseudo-posterior.Unlike standard variational inference which optimizes a lower bound on the log marginal likelihood, the new algorithms can be analyzed to provide loss guarantees on the predictions with the pseudo-posterior.Our criterion can be used to derive new sparse Gaussian process algorithms that have error guarantees applicable to various likelihoods.

Questo articolo utilizza l'analisi della loss di Lipschitz su uno spazio di ipotesi delimitato per derivare nuovi algoritmi di tipo ERM con forti garanzie di performance che possono essere applicati al modello GP sparso non coniugato.


In this paper, we propose a novel regularization method, RotationOut, for neural networks. Different from Dropout that handles each neuron/channel independently, RotationOut regards its input layer as an entire vector and introduces regularization by randomly rotating the vector. RotationOut can also be used in convolutional layers and recurrent layers with a small modification.We further use a noise analysis method to interpret the difference between RotationOut and Dropout in co-adaptation reduction. Using this method, we also show how to use RotationOut/Dropout together with Batch Normalization. Extensive experiments in vision and language tasks are conducted to show the effectiveness of the proposed method. Codes will be available.

Proponiamo un metodo di regolarizzazione per la rete neurale e un metodo di analisi del rumore

Questo articolo propone un nuovo metodo di regolarizzazione per mitigare il problema dell'overfitting delle reti neurali deep ruotando le caratteristiche con una matrice di rotazione casuale per ridurre il co-adattamento.

Questo articolo propone un nuovo metodo di regolarizzazione per il training delle reti neurali, che aggiunge neuroni di rumore in modo interdipendente.


Formulating the reinforcement learning (RL) problem in the framework of probabilistic inference not only offers a new perspective about RL, but also yields practical algorithms that are more robust and easier to train.While this connection between RL and probabilistic inference has been extensively studied in the single-agent setting, it has not yet been fully understood in the multi-agent setup.In this paper, we pose the problem of multi-agent reinforcement learning as the problem of performing inference in a particular graphical model.We model the environment, as seen by each of the agents, using separate but related Markov decision processes.We derive a practical off-policy maximum-entropy actor-critic algorithm that we call Multi-agent Soft Actor-Critic (MA-SAC) for performing approximate inference in the proposed model using variational inference.MA-SAC can be employed in both cooperative and competitive settings.Through experiments, we demonstrate that MA-SAC outperforms a strong baseline on several multi-agent scenarios.While MA-SAC is one resultant multi-agent RL algorithm that can be derived from the proposed probabilistic framework, our work provides a unified view of maximum-entropy algorithms in the multi-agent setting.

Un framework probabilistico per multi-agent reinforcement learning

Questo articolo propone un nuovo algoritmo chiamato Multi-Agent Soft Actor-Critic (MA-SAC) basato sull'algoritmo off-policy maximum-entropy actor critic Soft Actor-Critic (SAC)


Sorting input objects is an important step in many machine learning pipelines.However, the sorting operator is non-differentiable with respect to its inputs, which prohibits end-to-end gradient-based optimization.In this work, we propose NeuralSort, a general-purpose continuous relaxation of the output of the sorting operator from permutation matrices to the set of unimodal row-stochastic matrices, where every row sums to one and has a distinct argmax.This relaxation permits straight-through optimization of any computational graph involve a sorting operation.Further, we use this relaxation to enable gradient-based stochastic optimization over the combinatorially large space of permutations by deriving a reparameterized gradient estimator for the Plackett-Luce family of distributions over permutations.We demonstrate the usefulness of our framework on three tasks that require learning semantic orderings of high-dimensional objects, including a fully differentiable, parameterized extension of the k-nearest neighbors algorithm

Forniamo un rilassamento continuo all'operatore di ordinamento, consentendo un'ottimizzazione stocastica basata sul gradiente end-to-end.

L'articolo considera come ordinare un certo numero di elementi senza imparare esplicitamente e necessariamente i loro significati o valori reali e propone un metodo per eseguire l'ottimizzazione attraverso un rilassamento continuo.

Questo lavoro si basa su un'identità sum(top k) per derivare un campionatore differenziabile pathwise di matrici 'unimodal row stochastic'.

Introduce un rilassamento continuo dell'operatore di ordinamento per costruire un'ottimizzazione basata sul gradiente end-to-end e introduce un'estensione stocastica del suo metodo usando distribuzioni Placket-Luce e Monte Carlo.


Transferring knowledge across tasks to improve data-efficiency is one ofthe open key challenges in the area of global optimization algorithms.Readilyavailable algorithms are typically designed to be universal optimizers and, thus,often suboptimal for specific tasks.We propose a novel transfer learning method toobtain customized optimizers within the well-established framework of Bayesianoptimization, allowing our algorithm to utilize the proven generalizationcapabilities of Gaussian processes.Using reinforcement learning to meta-train anacquisition function (AF) on a set of related tasks, the proposed method learns toextract implicit structural information and to exploit it for improved data-efficiency.We present experiments on a sim-to-real transfer task as well as on several simulatedfunctions and two hyperparameter search problems.The results show that ouralgorithm (1) automatically identifies structural properties of objective functionsfrom available source tasks or simulations, (2) performs favourably in settings withboth scarse and abundant source data, and (3) falls back to the performance levelof general AFs if no structure is present.

Eseguiamo un transfer learning efficiente e flessibile nel framework dell'ottimizzazione bayesiana attraverso funzioni di acquisizione neurale meta-learned.

Gli autori presentano MetaBO che usa il reinforcement learning per apprendere tramite meta-learning la funzione di acquisizione per l'ottimizzazione bayesiana, mostrando una crescente sample-efficiency su nuovi task.

Gli autori propongono un'alternativa basata sul meta-learning alle funzioni di acquisizione standard (AF), in cui una rete neurale pretrained produce valori di acquisizione in funzione di feature scelte a mano.


We study the evolution of internal representations during deep neural network (DNN) training, aiming to demystify the compression aspect of the information bottleneck theory.The theory suggests that DNN training comprises a rapid fitting phase followed by a slower compression phase, in which the mutual information I(X;T) between the input X and internal representations T decreases.Several papers observe compression of estimated mutual information on different DNN models, but the true I(X;T) over these networks is provably either constant (discrete X) or infinite (continuous X).This work explains the discrepancy between theory and experiments, and clarifies what was actually measured by these past works.To this end, we introduce an auxiliary (noisy) DNN framework for which I(X;T) is a meaningful quantity that depends on the network's parameters.This noisy framework is shown to be a good proxy for the original (deterministic) DNN both in terms of performance and the learned representations.We then develop a rigorous estimator for I(X;T) in noisy DNNs and observe compression in various models.By relating I(X;T) in the noisy DNN to an information-theoretic communication problem, we show that compression is driven by the progressive clustering of hidden representations of inputs from the same class.Several methods to directly monitor clustering of hidden representations, both in noisy and deterministic DNNs, are used to show that meaningful clusters form in the T space.Finally, we return to the estimator of I(X;T) employed in past works, and demonstrate that while it fails to capture the true (vacuous) mutual information, it does serve as a measure for clustering.This clarifies the past observations of compression and isolates the geometric clustering of hidden representations as the true phenomenon of interest.

Le reti neurali deep deterministiche non scartano le informazioni, ma raggruppano i loro input.

Questo articolo fornisce un principio per esaminare la fase di compressione nelle reti neurali deep, fornendo uno stimatore di entropia supportato dalla teoria per stimare l'informazione mutua.


A central challenge in multi-agent reinforcement learning is the induction of coordination between agents of a team.In this work, we investigate how to promote inter-agent coordination using policy regularization and discuss two possible avenues respectively based on inter-agent modelling and synchronized sub-policy selection.We test each approach in four challenging continuous control tasks with sparse rewards and compare them against three baselines including MADDPG, a state-of-the-art multi-agent reinforcement learning algorithm.To ensure a fair comparison, we rely on a thorough hyper-parameter selection and training methodology that allows a fixed hyper-parameter search budget for each algorithm and environment.We consequently assess both the hyper-parameter sensitivity, sample-efficiency and asymptotic performance of each learning method.Our experiments show that the proposed methods lead to significant improvements on cooperative problems.We further analyse the effects of the proposed regularizations on the behaviors learned by the agents.

Proponiamo obiettivi di regolarizzazione per algoritmi RL multi-agente che favoriscono la coordinazione nei task cooperativi.

Questo articolo propone due metodi per indirizzare gli agenti verso l'apprendimento di comportamenti coordinati e valuta entrambi rigorosamente attraverso domini multi-agente di adeguata complessità.

Questo articolo propone due metodi basati su MADDPG per incoraggiare la collaborazione tra agenti MARL decentralizzati.