For instance, INTERPRETML #REFR leverages Jupyter Book [29] to incorporate in-notebook visualizations into its documentation, providing readers with an engaging way to learn about ML model explanations.

The notebook community has created a vibrant ecosystem to convert notebooks into a wide range of mediums.

This includes the ability for users to easily publish their notebooks containing interactive visualizations as slides #OTHEREFR , interactive books [29] , and dashboards #OTHEREFR .

Therefore, given the portability of notebooks, notebook VA tools have the potential to reach a more diverse audience.

Supernova: Design Strategies And Opportunities For Interactive Visualization In Computational Notebooks

Abstract: Computational notebooks such as Jupyter Notebook have become data scientists' de facto programming environments. Many visualization researchers and practitioners have developed interactive visualization tools that support notebooks. However, little is known about the appropriate design of visual analytics (VA) tools in notebooks. To bridge this critical research gap, we investigate the design strategies in this space by analyzing 161 notebook VA tools and their users' feedback. Our analysis encompasses 64 systems from academic papers and 103 systems sourced from a pool of 55k notebooks containing interactive visualizations that we obtain via scraping 8.6 million notebooks on GitHub. We also examine findings from 15 user studies and user feedback in 379 GitHub issues. Through this work, we identify unique design opportunities and considerations for future notebook VA tools, such as using and manipulating multimodal data in notebooks as well as balancing the degree of visualization-notebook integration. Finally, we develop SUPERNOVA, an open-source interactive tool to help researchers explore existing notebook VA tools and search for related work.

Interpretml: A Unified Framework For Machine Learning Interpretability

Abstract: InterpretML is an open-source Python package which exposes machine learning interpretability algorithms to practitioners and researchers. InterpretML exposes two types of interpretability - glassbox models, which are machine learning models designed for interpretability (ex: linear models, rule lists, generalized additive models), and blackbox explainability techniques for explaining existing systems (ex: Partial Dependence, LIME). The package enables practitioners to easily compare interpretability algorithms by exposing multiple methods under a unified API, and by having a built-in, extensible visualization platform. InterpretML also includes the first implementation of the Explainable Boosting Machine, a powerful, interpretable, glassbox model that can be as accurate as many blackbox models. The MIT licensed source code can be downloaded from this http URL.


In our work, we use a Gaussian prior distribution with zero mean and unit variance #REFR , and the MC sampling number of 50.

on Monday, Hist-D would calculate the mean µ t i and variance σ t i across all previously seen samples of node i at 8 a.m. of all Mondays.

Once the mean and variance for each time slot and node of interest are calculated, these two values can be used to construct the prediction interval for that slot and node with {µ t i -σ t i , µ t i +σ t i }.

Similarly, Hist-W would compute the weekly mean and variances across four weeks of a month to compute the PI.

Bayesian uncertainty quantification models the uncertainty in the model parameters using a likelihood function constructed by Bayesian modeling #OTHEREFR .

It also computes the data uncertainty by approximating the probability distribution over the model outputs through sampling and averaging over the resulting models.

Monte Carlo dropout #OTHEREFR models predictive distributions by randomly switching off neurons in DNNs during testing.

This generates different model outputs that can be interpreted as samples from a probabilistic distribution, allowing MC dropout to estimate the prediction probability.

In our work, we added a dropout layer with a rate of 0.3 after the last hidden layer of the base traffic forecasting model and used a sample number of 50.

Deep Quantile Regression (DQR) can also estimate PIs #OTHEREFR , #OTHEREFR .

Unlike conventional methods that minimize the averaged residual errors, DQR calculates the prediction errors at a specific quantile of the distribution.

Adaptive Modeling Of Uncertainties For Traffic Forecasting

Abstract: Deep neural networks (DNNs) have emerged as a dominant approach for developing traffic forecasting models. These models are typically trained to minimize error on averaged test cases and produce a single-point prediction, such as a scalar value for traffic speed or travel time. However, singlepoint predictions fail to account for prediction uncertainty that is critical for many transportation management scenarios, such as determining the best-or worst-case arrival time. We present QUANTRAFFIC, a generic framework to enhance the capability of an arbitrary DNN model for uncertainty modeling. QUANTRAFFIC requires little human involvement and does not change the base DNN architecture during deployment. Instead, it automatically learns a standard quantile function during the DNN model training to produce a prediction interval for the single-point prediction. The prediction interval defines a range where the true value of the traffic prediction is likely to fall. Furthermore, QUANTRAFFIC develops an adaptive scheme that dynamically adjusts the prediction interval based on the location and prediction window of the test input. We evaluated QUANTRAFFIC by applying it to five representative DNN models for traffic forecasting across seven public datasets. We then compared QUANTRAFFIC against five uncertainty quantification methods. Compared to the baseline uncertainty modeling techniques, QUANTRAFFIC with base DNN architectures delivers consistently better and more robust performance than the existing ones on the reported datasets.

Weight Uncertainty In Neural Networks

Abstract: We introduce a new, efficient, principled and backpropagation-compatible algorithm for learning a probability distribution on the weights of a neural network, called Bayes by Backprop. It regularises the weights by minimising a compression cost, known as the variational free energy or the expected lower bound on the marginal likelihood. We show that this principled kind of regularisation yields comparable performance to dropout on MNIST classification. We then demonstrate how the learnt uncertainty in the weights can be used to improve generalisation in non-linear regression problems, and how this weight uncertainty can be used to drive the exploration-exploitation trade-off in reinforcement learning.

unit variance
Gaussian prior distribution


In particular, our multi-model bijective network is designed as multi-scale architecture adopted from #REFR where each scale includes multiple steps of the flow.

In this section, we first review datasets, implementation, and evaluation benchmarks.

Then, the ablation studies analyze the effectiveness of our proposed approach.

Finally, we compare our SOTA results with prior UDA approaches. #OTHEREFR with a MiT-B4 encoder #OTHEREFR .

Following the UAV protocol of #OTHEREFR , the image size is set to 1024 × 1024.

Every single flow step injected by the domain information is designed as a stack of AcNorm, Invertible 1 × 1 Convolution, and Residual-style Affine Coupling Layer #OTHEREFR , #OTHEREFR , #OTHEREFR .

The number of scales and flows in our experiments are set to 4 and 32, respectively.

The entire framework is optimized by the SGD optimizer on four 48GB-VRAM GPUs, where the batch size of each GPU is set to 8 and the base learning rate is set to 2.5×10 −4 .

To increase the diversity of training data, several data augmentation techniques #OTHEREFR , #OTHEREFR are adopted in the training process.

Crovia: Seeing Drone Scenes From Car Perspective Via Cross-View Adaptation

Abstract: Understanding semantic scene segmentation of urban scenes captured from the Unmanned Aerial Vehicles (UAV) perspective plays a vital role in building a perception model for UAV. With the limitations of large-scale densely labeled data, semantic scene segmentation for UAV views requires a broad understanding of an object from both its top and side views. Adapting from well-annotated autonomous driving data to unlabeled UAV data is challenging due to the cross-view differences between the two data types. Our work proposes a novel Cross-View Adaptation (CROVIA) approach to effectively adapt the knowledge learned from on-road vehicle views to UAV views. First, a novel geometry-based constraint to cross-view adaptation is introduced based on the geometry correlation between views. Second, cross-view correlations from image space are effectively transferred to segmentation space without any requirement of paired on-road and UAV view data via a new Geometry-Constraint Cross-View (GeiCo) loss. Third, the multi-modal bijective networks are introduced to enforce the global structural modeling across views. Experimental results on new cross-view adaptation benchmarks introduced in this work, i.e., SYNTHIA → UAVID and GTA5 → UAVID, show the State-of-the-Art (SOTA) performance of our approach over prior adaptation methods.

Alignflow: Cycle Consistent Learning From Multiple Domains Via Normalizing Flows

Abstract: Given datasets from multiple domains, a key challenge is to efficiently exploit these data sources for modeling a target domain. Variants of this problem have been studied in many contexts, such as cross-domain translation and domain adaptation. We propose AlignFlow, a generative modeling framework that models each domain via a normalizing flow. The use of normalizing flows allows for a) flexibility in specifying learning objectives via adversarial training, maximum likelihood estimation, or a hybrid of the two methods; and b) learning and exact inference of a shared representation in the latent space of the generative model. We derive a uniform set of conditions under which AlignFlow is marginally-consistent for the different learning objectives. Furthermore, we show that AlignFlow guarantees exact cycle consistency in mapping datapoints from a source domain to target and back to the source domain. Empirically, AlignFlow outperforms relevant baselines on image-to-image translation and unsupervised domain adaptation and can be used to simultaneously interpolate across the various domains using the learned representation.


Optimal Experimental Design For Biology Network Recovery

Hauser and Bühlmann #REFR similarly propose a utility function based on the number of oriented edges of a skeleton graph.

Another way to choose the optimal intervention is to choose the intervention that leads to the maximal number of oriented edges. Ness et al.

#OTHEREFR use optimal experimental design to recover protein signaling networks #OTHEREFR .

They use a utility function based on the expected information gain of an intervention given the observational MEC and other interventions in the batch. This algorithm, however, has factorial dependence on batch size. Ghassami et al.

#OTHEREFR use the expected number of oriented edges of an essential graph as the utility function.

The essential graph of the ground truth network is first estimated using a constraint based method like the PC algorithm.

Causal Discovery And Optimal Experimental Design For Genome-Scale Biological Network Recovery

Abstract: Causal discovery of genome-scale networks is important for identifying pathways from genes to observable traits-e.g. differences in cell function, disease, drug resistance and others. Causal learners based on graphical models rely on interventional samples to orient edges in the network. However, these models have not been shown to scale up the size of the genome, which are on the order of 10 3-10 4 genes. We introduce a new learner, SP-GIES, that jointly learns from interventional and observational datasets and achieves almost 4x speedup against an existing learner for 1,000 node networks. SP-GIES achieves an AUC-PR score of 0.91 on 1,000 node networks, and scales up to 2,000 node networks-this is 4x larger than existing works. We also show how SP-GIES improves downstream optimal experimental design strategies for selecting interventional experiments to perform on the system. This is an important step forward in realizing causal discovery at scale via autonomous experimental design.

Two Optimal Strategies For Active Learning Of Causal Models From Interventional Data

Abstract: From observational data alone, a causal DAG is only identifiable up to Markov equivalence. Interventional data generally improves identifiability; however, the gain of an intervention strongly depends on the intervention target, that is, the intervened variables. We present active learning (that is, optimal experimental design) strategies calculating optimal interventions for two different learning goals. The first one is a greedy approach using single-vertex interventions that maximizes the number of edges that can be oriented after each intervention. The second one yields in polynomial time a minimum set of targets of arbitrary size that guarantees full identifiability. This second approach proves a conjecture of Eberhardt (2008) indicating the number of unbounded intervention targets which is sufficient and in the worst case necessary for full identifiability. In a simulation study, we compare our two active learning approaches to random interventions and an existing approach, and analyze the influence of estimation errors on the overall performance of active learning.


Ii. Fap Channel Model In Diffusive Molecular Communication Systems

The value of D is determined by the temperature, the fluid viscosity, and the molecule's Stokes radius, see #REFR .

Readers can refer to #OTHEREFR or #OTHEREFR for more details on diffusion channel modeling.

The PDE approach treats the distribution of particles as a continuous function/data and uses PDE to capture its evolution.

Namely, if we denote the concentration function by c (r, t; r 0 ), the diffusion channel can be modeled by a diffusion-advection equation:

where r 0 is the point where the diffusion starts, v is the velocity field of the fluid medium which is assumed to be incompressible, ∇ and ∇ 2 are the gradient and the Laplacian operators, respectively, and D is the diffusion coefficient.

On the other hand, if we treat each particle individually, we can use the SDE model to capture its random trajectory X t .

A suitable SDE model for the trajectory is the Itô diffusion process.

Let us recap briefly that an Itô diffusion in Euclidean space R d is a stochastic process X t satisfying an SDE of the form

where B t is an d-dimensional standard Brownian motion.

On Vertically-Drifted First Arrival Position Distribution In Diffusion Channels

Abstract: Recent studies show that stable distributions are successful in modeling heavy-tailed or impulsive noise. Investigation of the stability of a probability distribution can be greatly facilitated if the corresponding characteristic function (CF) has a closed-form expression. We explore a new family of distribution called the Vertically-Drifted First Arrival Position (VDFAP) distribution, which can be viewed as a generalization of symmetric alpha-stable (SαS) distribution with stability parameter α = 1. In addition, VDFAP distribution has a clear physical interpretation when we consider first-hitting problems of particles following Brownian motion with a driving drift. Inspired by the Fourier relation between the probability density function and CF of Student's t-distribution, we extract an integral representation for the VDFAP probability density function. Then, we exploit the Hankel transform to derive a closed-form expression for the CF of VDFAP. From the CF, we discover that VDFAP possesses some interesting stability properties, which are in a weaker form than SαS. This calls for a generalization of the theory on alpha-stable distributions.

A Comprehensive Survey Of Recent Advancements In Molecular Communication

Abstract: With much advancement in the field of nanotechnology, bioengineering, and synthetic biology over the past decade, microscales and nanoscales devices are becoming a reality. Yet the problem of engineering a reliable communication system between tiny devices is still an open problem. At the same time, despite the prevalence of radio communication, there are still areas where traditional electromagnetic waves find it difficult or expensive to reach. Points of interest in industry, cities, and medical applications often lie in embedded and entrenched areas, accessible only by ventricles at scales too small for conventional radio waves and microwaves, or they are located in such a way that directional high frequency systems are ineffective. Inspired by nature, one solution to these problems is molecular communication (MC), where chemical signals are used to transfer information. Although biologists have studied MC for decades, it has only been researched for roughly 10 year from a communication engineering lens. Significant number of papers have been published to date, but owing to the need for interdisciplinary work, much of the results are preliminary. In this survey, the recent advancements in the field of MC engineering are highlighted. First, the biological, chemical, and physical processes used by an MC system are discussed. This includes different components of the MC transmitter and receiver, as well as the propagation and transport mechanisms. Then, a comprehensive survey of some of the recent works on MC through a communication engineering lens is provided. The survey ends with a technology readiness analysis of MC and future research directions.

fluid viscosity
molecule's Stokes radius


XLNet #REFR was based on an auto-regressive model, which predicts future behavior based on past, and used a Transformer-XL.

Fine-tuning means to further train the pre-trained BERT model using our data.

Section 5.1 includes the details of our BERT model and its hyperparameters.

There have been numerous models extending BERT. In our experiments, we used RoBERTa and XLNet.

As main differences from BERT, RoBERTa (Robustly optimized BERT approach) #OTHEREFR removed the NSP and replaced the static masking (in MLM) of BERT, with dynamic masking.

In summary, RoBERTa has been shown to be more robust than BERT, it modified some of BERT, and it was trained using more data.

XLNet also introduced permutation language modeling, where all tokens (not only masked tokens) are predicted in random order, rather than sequential.

Deep Learning For Opinion Mining And Topic Classification Of Course Reviews

Abstract: Student opinions for a course are important to educators and administrators, regardless of the type of the course or the institution. Reading and manually analyzing open-ended feedback becomes infeasible for massive volumes of comments at institution level or online forums. In this paper, we collected and pre-processed a large number of course reviews publicly available online. We applied machine learning techniques with the goal to gain insight into student sentiments and topics. Specifically, we utilized current Natural Language Processing (NLP) techniques, such as word embeddings and deep neural networks, and state-of-the-art BERT (Bidirectional Encoder Representations from Transformers), RoBERTa (Robustly optimized BERT approach) and XLNet (Generalized Autoregression Pre-training). We performed extensive experimentation to compare these techniques versus traditional approaches. This comparative study demonstrates how to apply modern machine learning approaches for sentiment polarity extraction and topic-based classification utilizing course feedback. For sentiment polarity, the top model was RoBERTa with 95.5% accuracy and 84.7% F1-macro, while for topic classification, an SVM (Support Vector Machine) was the top classifier with 79.8% accuracy and 80.6% F1-macro. We also provided an in-depth exploration of the effect of certain hyperparameters on the model performance and discussed our observations. These findings can be used by institutions and course providers as a guide for analyzing their own course feedback using NLP models towards self-evaluation and improvement.

Xlnet: Generalized Autoregressive Pretraining For Language Understanding

Abstract: With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language modeling. However, relying on corrupting the input with masks, BERT neglects dependency between the masked positions and suffers from a pretrain-finetune discrepancy. In light of these pros and cons, we propose XLNet, a generalized autoregressive pretraining method that (1) enables learning bidirectional contexts by maximizing the expected likelihood over all permutations of the factorization order and (2) overcomes the limitations of BERT thanks to its autoregressive formulation. Furthermore, XLNet integrates ideas from Transformer-XL, the state-of-the-art autoregressive model, into pretraining. Empirically, under comparable experiment setting, XLNet outperforms BERT on 20 tasks, often by a large margin, including question answering, natural language inference, sentiment analysis, and document ranking.


We have a similar finding in Macron (2017) and Yellow Vests (2019) where the missing tweets are against Macron such as #MacronDemission which means "Macron Resign", and #MacronLeaks, a coordinated disinformation campaign #REFR or supportive of his opponent Marine Le Pen.

Topics To answer RQ3, we performed a qualitative topic analysis on the missing tweets to better understand the content the studies may not be able to capture.

To do that we, compared the content in the missing tweets with the recollected datasets using word shift graphs (See Section 3).

For brevity, we only present the results with 15 datasets where we observe clear differences and only the top 20 n-grams.

We observe that most of the missing tweets contain slogans against the topic.

For instance, missing tweets in Biden (2020), Hillary (2016), and in Pro-Hillary (2016) appear to be sourced from users in favor of Donald Trump as the common hashtags include #MAGA, #Trump2020, #NeverHillary, and #CrookedHillary.

This suggests that data persistence may affect studies analyzing such counter-groups.

We observe that the missing tweets in the datasets related to Trump politics such as Kavanaugh (2018) and Paris Agreement (2018) also contain tweets that are supportive of the decisions, such as #ConfirmKavanaughNow and #AmericaFirst.

The missing tweets in the conspiracy theory-related datasets are supportive of the theories (e.g., #GreatAwakening, #Pedogate) while the tweets in the recollected datasets contain the 2-gram "conspiracy-theory", suggesting that the remaining tweets are mostly critical of the theories.

Interestingly, the recollected Gun Control dataset is more likely to contain the hashtag #GunViolence, which is a more extreme version of the hashtag #GunControl.

We also find that the missing tweets in the Netflix dataset contain the 2-grams "Full-access", "Netflix-premium", and "watch-netflix.

The Impact Of Data Persistence Bias On Social Media Studies

Abstract: Social media studies often collect data retrospectively to analyze public opinion. Social media data may decay over time and such decay may prevent the collection of the complete dataset. As a result, the collected dataset may differ from the complete dataset and the study may suffer from data persistence bias. Past research suggests that the datasets collected retrospectively are largely representative of the original dataset in terms of textual content. However, no study analyzed the impact of data persistence bias on social media studies such as those focusing on controversial topics. In this study, we analyze the data persistence and the bias it introduces on the datasets of three types: controversial topics, trending topics, and framing of issues. We report which topics are more likely to suffer from data persistence among these datasets. We quantify the data persistence bias using the change in political orientation, the presence of potentially harmful content and topics as measures. We found that controversial datasets are more likely to suffer from data persistence and they lean towards the political left upon recollection. The turnout of the data that contain potentially harmful content is significantly lower on non-controversial datasets. Overall, we found that the topics promoted by right-aligned users are more likely to suffer from data persistence. Account suspensions are the primary factor contributing to data removals, if not the only one. Our results emphasize the importance of accounting for the data persistence bias by collecting the data in real time when the dataset employed is vulnerable to data persistence bias.

Disinformation And Social Bot Operations In The Run Up To The 2017 French Presidential Election

Abstract: Recent accounts from researchers, journalists, as well as federal investigators, reached a unanimous conclusion: social media are systematically exploited to manipulate and alter public opinion. Some disinformation campaigns have been coordinated by means of bots, social media accounts controlled by computer scripts that try to disguise themselves as legitimate human users. In this study, we describe one such operation that occurred in the run up to the 2017 French presidential election. We collected a massive Twitter dataset of nearly 17 million posts, posted between 27 April and 7 May 2017 (Election Day). We then set to study the MacronLeaks disinformation campaign: By leveraging a mix of machine learning and cognitive behavioral modeling techniques, we separated humans from bots, and then studied the activities of the two groups independently, as well as their interplay. We provide a characterization of both the bots and the users who engaged with them, and oppose it to those users who didn’t. Prior interests of disinformation adopters pinpoint to the reasons of scarce success of this campaign: the users who engaged with MacronLeaks are mostly foreigners with pre-existing interest in alt-right topics and alternative news media, rather than French users with diverse political views. Concluding, anomalous account usage patterns suggest the possible existence of a black market for reusable political disinformation bots.


The baseline model, XLM-RoBERTa-base (XLM-R-base) #REFR , was trained and fine-tuned by minimizing the log-likelihood loss.

The starter kit for sub-task 1 can be found in the NL4Opt repository 2 .

As part of the pilot study, we reported 3 the baseline model's performance on the test set when evaluated on the source domain, target domain, and entire test set for all entity types (i.e., constraint direction, limit, etc.).

Based on this preliminary analysis, the objective name was the most difficult to identify potentially due to its ambiguity.

We expect the greatest improvements would arise from methods that are capable of accurately recognizing the objective names and their spans.

Evaluation: This baseline achieved an F1 score of 0.906 on the test split.

Nl4Opt Competition: Formulating Optimization Problems Based On Their Natural Language Descriptions

Abstract: The Natural Language for Optimization (NL4Opt) Competition was created to investigate methods of extracting the meaning and formulation of an optimization problem based on its text description. Specifically, the goal of the competition is to increase the accessibility and usability of optimization solvers by allowing non-experts to interface with them using natural language. We separate this challenging goal into two sub-tasks: (1) recognize and label the semantic entities that correspond to the components of the optimization problem; (2) generate a meaning representation (i.e. a logical form) of the problem from its detected problem entities. The first task aims to reduce ambiguity by detecting and tagging the entities of the optimization problems. The second task creates an intermediate representation of the linear programming (LP) problem that is converted into a format that can be used by commercial solvers. In this report, we present the LP word problem dataset and shared tasks for the NeurIPS 2022 competition. Furthermore, we investigate and compare the performance of the ChatGPT large language model against the winning solutions. Through this competition, we hope to bring interest towards the development of novel machine learning applications and datasets for optimization modeling.

Unsupervised Cross-Lingual Representation Learning At Scale

Abstract: This paper shows that pretraining multilingual language models at scale leads to significant performance gains for a wide range of cross-lingual transfer tasks. We train a Transformer-based masked language model on one hundred languages, using more than two terabytes of filtered CommonCrawl data. Our model, dubbed XLM-R, significantly outperforms multilingual BERT (mBERT) on a variety of cross-lingual benchmarks, including +13.8% average accuracy on XNLI, +12.3% average F1 score on MLQA, and +2.1% average F1 score on NER. XLM-R performs particularly well on low-resource languages, improving 11.8% in XNLI accuracy for Swahili and 9.2% for Urdu over the previous XLM model. We also present a detailed empirical evaluation of the key factors that are required to achieve these gains, including the trade-offs between (1) positive transfer and capacity dilution and (2) the performance of high and low resource languages at scale. Finally, we show, for the first time, the possibility of multilingual modeling without sacrificing per-language performance; XLM-Ris very competitive with strong monolingual models on the GLUE and XNLI benchmarks. We will make XLM-R code, data, and models publicly available.


B. Constraint-Based Optimization Of Deployments On Gpu

To the best of our knowledge, this is the first to consider these constraints for deployments of DNN models on GPUs, compared with previous arts, e.g., #REFR .

These constraints are usually ignored by designers, which wastes lots of optimization workloads.

For examples, for T ∈ [T r , T sm ], these T values are legal while on-board resources are not fully utilized and the system parallelism can be further improved.

With the above constraints, the feasible domain of the block sizes is shrunken significantly.

The visualization view of the constraints are shown in Fig. 7 .

With the target of minimizing the inference latency, we tend to build a regression model with respect to candidate values of nx, ny, and nz.

The key challenge is that the clear form is the objective function is unknown because of the invisible execution process of GPU and CUDA programming Fig. 7 The visualized solution space.

The solution points below the dotted points are legal configurations.

Bayesian optimization is adopted in this paper as the searching algorithm to search the optimal configuration of the blocks with Gaussian process (GP) model utilized as the surrogate model #OTHEREFR .

A High-Performance Accelerator For Super-Resolution Processing On Embedded Gpu

Abstract: Recent years have witnessed impressive progress in super-resolution (SR) processing. However, its real-time inference requirement sets a challenge not only for the model design but also for the on-chip implementation. In this paper, we implement a full-stack SR acceleration framework on embedded GPU devices. The special dictionary learning algorithm used in SR models was analyzed in detail and accelerated via a novel dictionary selective strategy. Besides, the hardware programming architecture together with the model structure is analyzed to guide the optimal design of computation kernels to minimize the inference latency under the resource constraints. With these novel techniques, the communication and computation bottlenecks in the deep dictionary learning-based SR models are tackled perfectly. The experiments on the edge embedded NVIDIA NX and 2080Ti show that our method outperforms the state-of-the-art NVIDIA TensorRT significantly, and can achieve real-time performance.

Tvm: An Automated End-To-End Optimizing Compiler For Deep Learning

Abstract: There is an increasing need to bring machine learning to a wide diversity of hardware devices. Current frameworks rely on vendor-specific operator libraries and optimize for a narrow range of server-class GPUs. Deploying workloads to new platforms -- such as mobile phones, embedded devices, and accelerators (e.g., FPGAs, ASICs) -- requires significant manual effort. We propose TVM, a compiler that exposes graph-level and operator-level optimizations to provide performance portability to deep learning workloads across diverse hardware back-ends. TVM solves optimization challenges specific to deep learning, such as high-level operator fusion, mapping to arbitrary hardware primitives, and memory latency hiding. It also automates optimization of low-level programs to hardware characteristics by employing a novel, learning-based cost modeling method for rapid exploration of code optimizations. Experimental results show that TVM delivers performance across hardware back-ends that are competitive with state-of-the-art, hand-tuned libraries for low-power CPU, mobile GPU, and server-class GPUs. We also demonstrate TVM's ability to target new accelerator back-ends, such as the FPGA-based generic deep learning accelerator. The system is open sourced and in production use inside several major companies.


Re-Lie #REFR performs normalizing flow in R 3 and transform a Euclidean vector into an element of SO(3).

However, non of the grasping methods can generalize to a large number of objects under raw vision input.

In comparison, our goal-conditioned grasp execution method is the first vision-based pipeline to achieve universal generalization by leveraging a teacher-student distillation trick, object curriculum learning, and state canonicalization.

Normalizing Flow Normalizing flow is a powerful technique for modeling highly complex distributions, suiting our needs for grasp proposal generation.

Building normalizing flow in the Euclidean space has been well studied #OTHEREFR .

However, it is much harder to build normalizing flow in a non-Euclidean space such as SO(3).

However, this practice introduces discontinuity to the flow, because it is equivalent to using the axis angle representation.

Therefore, our grasp proposal module decouples rotation from translation and joint angles, and model these distributions separately with IPDF #OTHEREFR and normalizing flow.

Unidexgrasp: Universal Robotic Dexterous Grasping Via Learning Diverse Proposal Generation And Goal-Conditioned Policy

Abstract: Figure 1. UniDexGrasp via grasp proposal generation and goal-conditioned execution. Left (grasp proposals): each figure shows for an object we generate diverse and high-quality grasp poses that vary greatly in rotation, translation and articulation states; right (grasp execution): given a grasp goal pose, our highly generalizable goal-conditioned grasping policy can grasp the object in the way specified by the goal, as shown in the green and blue trajectories and their corresponding goals.

Reparameterizing Distributions On Lie Groups

Abstract: Reparameterizable densities are an important way to learn probability distributions in a deep learning setting. For many distributions it is possible to create low-variance gradient estimators by utilizing a `reparameterization trick'. Due to the absence of a general reparameterization trick, much research has recently been devoted to extend the number of reparameterizable distributional families. Unfortunately, this research has primarily focused on distributions defined in Euclidean space, ruling out the usage of one of the most influential class of spaces with non-trivial topologies: Lie groups. In this work we define a general framework to create reparameterizable densities on arbitrary Lie groups, and provide a detailed practitioners guide to further the ease of usage. We demonstrate how to create complex and multimodal distributions on the well known oriented group of 3D rotations, SO(3), using normalizing flows. Our experiments on applying such distributions in a Bayesian setting for pose estimation on objects with discrete and continuous symmetries, showcase their necessity in achieving realistic uncertainty estimates.


This result is still significantly less than the state-of-the-art results across many other NLP tasks though (e.g., for the SQuAD dataset #REFR ), hinting that there is still room for significant improvement here.

As expected, due to the generative nature of the majority of the implicit examples, the extractive NPR approach performed significantly better on explicit examples (F1 score of 0.150 for implicit examples versus 0.625 for explicit examples).

In order to gain an insight into the contribution of individual features, we report the mean results across all configurations for when this feature is set and when it is unset.

The mean F1-scores for different options across all configurations in which they are enabled were 0.44 with word vectors, 0.39 without word vectors, 0.42 with NNAN candidate generation, 0.41 with noun chunk candidate generation, and 0.45, 0.42 and 0.38 for a negative sampling factor of 1, 2 and 4 respectively.

Thus, on average, across all configurations the best performing ones used word vectors, NNAN candidate generation and downsampled negative to positive samples to a 1:1 ratio.

The NPR approach clearly performs well for explicit examples, with the top configuration achieving an F1-Score of 0.625.

The fact that the variations of the NPR approach that we evaluated on had little impact on the results achieved suggests that a more distinct strategy may be required, rather than a variation of the SOTA.

Further, the contrast in performance on explicit versus implicit examples, illustrates the importance of considering non-span or grounded generative examples in a realistic evaluation of the MIDR task.

Whilst, it is clear that the extractive NPR approach will achieve a zero score for this class of examples in the Exact Match metric, the top F1-Score of 0.15, is also significantly lower than for explicit examples.

Also, the top performing configurations for the grouping All Identifiers differ from those for the grouping All Descriptions showing the importance of considering examples without a description in MIDR datasets.

What Makes A Good Dataset For Symbol Description Reading?

Abstract: The usage of mathematical formulas as concise representations of a document's key ideas is common practice. Correctly interpreting these formulas, by identifying mathematical symbols and extracting their descriptions, is an important task in document understanding. This paper makes the following contributions to the mathematical identifier description reading (MIDR) task: (i) introduces the Math Formula Question Answering Dataset (MFQuAD) with 7508 annotated identifier occurrences; (ii) describes novel variations of the noun phrase ranking approach for the MIDR task; (iii) reports experimental results for the SOTA noun phrase ranking approach and our novel variations of the approach, providing problem insights and a performance baseline; (iv) provides a position on the features that make an effective dataset for the MIDR task.

Know What You Don'T Know: Unanswerable Questions For Squad

Abstract: Extractive reading comprehension systems can often locate the correct answer to a question in a context document, but they also tend to make unreliable guesses on questions for which the correct answer is not stated in the context. Existing datasets either focus exclusively on answerable questions, or use automatically generated unanswerable questions that are easy to identify. To address these weaknesses, we present SQuAD 2.0, the latest version of the Stanford Question Answering Dataset (SQuAD). SQuAD 2.0 combines existing SQuAD data with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD 2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering. SQuAD 2.0 is a challenging natural language understanding task for existing models: a strong neural system that gets 86% F1 on SQuAD 1.1 achieves only 66% F1 on SQuAD 2.0.


The results show that Pred-NBV is able to outperform the baselines significantly using large scale models from the Shapenet #REFR dataset and with real world 3D LiDAR data.

In this section we present results on the evaluation of the Pred-NBV pipeline.

We start with a qualitative example followed by a discussion of the performance of the individual modules against respective baseline methods.

Figure 2 show the reconstructed point cloud of a C17 airplane the path followed by a UAV and in AirSim #OTHEREFR .

We create candidate poses on three concentric rings at different heights around the center of the partially observed point cloud.

The candidate poses change as more of the object is visible.

As shown in Figure 6 , Pred-NBV is able to observe more points than the NBV planner without prediction in the same time budget.

Pred-Nbv: Prediction-Guided Next-Best-View Planning For 3D Object Reconstruction

Abstract: Prediction-based active perception has shown the potential to improve the navigation efficiency and safety of the robot by anticipating the uncertainty in the unknown environment. The existing works for 3D shape prediction make an implicit assumption about the partial observations and therefore cannot be used for real-world planning and do not consider the control effort for next-best-view planning. We present Pred-NBV, a realistic object shape reconstruction method consisting of PoinTr-C, an enhanced 3D prediction model trained on the ShapeNet dataset, and an information and control effort-based next-best-view method to address these issues. Pred-NBV shows an improvement of 25.46% in object coverage over the traditional methods in the AirSim simulator, and performs better shape completion than PoinTr, the stateof-the-art shape completion model, even on real data obtained from a Velodyne 3D LiDAR mounted on DJI M600 Pro.

Shapenet: An Information-Rich 3D Model Repository

Abstract: We present ShapeNet: a richly-annotated, large-scale repository of shapes represented by 3D CAD models of objects. ShapeNet contains 3D models from a multitude of semantic categories and organizes them under the WordNet taxonomy. It is a collection of datasets providing many semantic annotations for each 3D model such as consistent rigid alignments, parts and bilateral symmetry planes, physical sizes, keywords, as well as other planned annotations. Annotations are made available through a public web-based interface to enable data visualization of object attributes, promote data-driven geometric analysis, and provide a large-scale quantitative benchmark for research in computer graphics and vision. At the time of this technical report, ShapeNet has indexed more than 3,000,000 models, 220,000 models out of which are classified into 3,135 categories (WordNet synsets). In this report we describe the ShapeNet effort as a whole, provide details for all currently available datasets, and summarize future plans.


Patil et al.'s #REFR work on depth prediction and completion demonstrates that recurrent modules enforce consistency and improve accuracy even without special loss functions.

But it does have the advantage of not relying on camera poses or external structures. Lai et al. #OTHEREFR and Cao et al.

#OTHEREFR pair a warping loss with a convolutional recurrent module which allows the network to learn temporal affinities more effectively.

In fact, the use of recurrent structures is another way of enforcing temporal consistency in video tasks #OTHEREFR . Zhang et al.

#OTHEREFR propagate spatio-temporal depth information across frames using a novel convolutional LSTM module which is trained using an adversarial loss.

While this loss does not have any explicit temporal constraint like the warping loss, it is nonetheless shown to improve depth consistency.

Convolutional LSTMs, however, can be difficult to train due to their large memory requirements.

Depth Fusion: Fusion methods #OTHEREFR achieve scene reconstruction by blending weighted signed distance field (SDF) volumes for each frame.

The use of SDF volumes makes them scale poorly to large scenes and high resolutions. Keller et al. #OTHEREFR and Lefloch et al. #OTHEREFR propose point clouds as an alternative to SDFs.

Traditional fusion methods, however, are not directly applicable to depth reconstruction as their focus is geometric reconstruction over multiple frames.

Completeness, in the sense of reconstructing each visible point in a frame, is not a requirement.

Temporally Consistent Online Depth Estimation Using Point-Based Fusion

Abstract: Depth estimation is an important step in many computer vision problems such as 3D reconstruction, novel view synthesis, and computational photography. Most existing work focuses on depth estimation from single frames. When applied to videos, the result lacks temporal consistency, showing flickering and swimming artifacts. In this paper we aim to estimate temporally consistent depth maps of video streams in an online setting. This is a difficult problem as future frames are not available and the method must choose between enforcing consistency and correcting errors from previous estimations. The presence of dynamic objects further complicates the problem. We propose to address these challenges by using a global point cloud that is dynamically updated each frame, along with a learned fusion approach in image space. Our approach encourages consistency while simultaneously allowing updates to handle errors and dynamic objects. Qualitative and quantitative results show that our method achieves state-of-the-art quality for consistent video depth estimation.

Don'T Forget The Past: Recurrent Depth Estimation From Monocular Video

Abstract: Autonomous cars need continuously updated depth information. Thus far, the depth is mostly estimated independently for a single frame at a time, even if the method starts from video input. Our method produces a time series of depth maps, which makes it an ideal candidate for online learning approaches. In particular, we put three different types of depth estimation (supervised depth prediction, self-supervised depth prediction, and self-supervised depth completion) into a common framework. We integrate the corresponding networks with a convolutional LSTM such that the spatiotemporal structures of depth across frames can be exploited to yield a more accurate depth estimation. Our method is flexible. It can be applied to monocular videos only or be combined with different types of sparse depth patterns. We carefully study the architecture of the recurrent network and its training strategy. We are first to successfully exploit recurrent networks for real-time self-supervised monocular depth estimation and completion. Extensive experiments show that our recurrent method outperforms its image-based counterpart consistently and significantly in both self-supervised scenarios. It also outperforms previous depth estimation methods of the three popular groups.


The rest of the model's structure follows the classic U-Net architecture #REFR with a series of 2D-Conv and Transpose layer blocks plus skip connection.

The last layer of each block diminishes by a factor equal to the stride value in the time dimension.

The stride values are 2, 3, and 5, respectively, for each block.

The output of the 4D-Conv layers is a tensor where the temporal dimension has been squeezed and reduced; information are extrapolated from the temporal dimension.

Thus the output resulting from the 4D-Conv layers contains only three dimensions (X × Y × Z) plus the channel dimension.

3D-Conv layers are implemented to reduce the depth dimension Z and produce a 2D vector (X × Y ) plus the channel dimension.

All 2D-Conv Transpose layers utilize a kernel with shape (2 × 2).

Every max pooling layer, 3D-Conv, and 2D-Conv layers use the same parameters described in Sec. 4.2.

Two MonteCarlo dropout layers #OTHEREFR are added at the end of the 4D and 2D Convolution blocks. The rate was set to 50%.

These layers were added to reduce uncertainties in the final predictions.

The last convolution layer has a kernel of (1 × 1) and a Softmax activation function to produce a probability score for every class.

Exploiting 4D Ct Perfusion For Segmenting Infarcted Areas In Patients With Suspected Acute Ischemic Stroke

Abstract: Precise and fast prediction methods for ischemic areas (core and penumbra) in acute ischemic stroke (AIS) patients are of significant clinical interest: they play an essential role in improving diagnosis and treatment planning. Computed Tomography (CT) scan is one of the primary modalities for early assessment in patients with suspected AIS. CT Perfusion (CTP) is often used as a primary assessment to determine stroke location, severity, and volume of ischemic lesions. Current automatic segmentation methods for CTP mostly use already processed 3D color maps conventionally used for visual assessment by radiologists as input. Alternatively, the raw CTP data is used on a slice-by-slice basis as 2D+time input, where the spatial information over the volume is ignored. In this paper, we investigate different methods to utilize the entire 4D CTP as input to fully exploit the spatio-temporal information. This leads us to propose a novel 4D convolution layer. Our comprehensive experiments on a local dataset comprised of 152 patients divided into three groups show that our proposed models generate more precise results than other methods explored. A Dice Coefficient of 0.70 and 0.45 is achieved for All authors are with the BioMedical Data analysis group (https://www.uis.no/en/ bmdlab)

U-Net: Convolutional Networks For Biomedical Image Segmentation

Abstract: There is large consent that successful training of deep networks requires many thousand annotated training samples. In this paper, we present a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently. The architecture consists of a contracting path to capture context and a symmetric expanding path that enables precise localization. We show that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks. Using the same network trained on transmitted light microscopy images (phase contrast and DIC) we won the ISBI cell tracking challenge 2015 in these categories by a large margin. Moreover, the network is fast. Segmentation of a 512x512 image takes less than a second on a recent GPU. The full implementation (based on Caffe) and the trained networks are available at http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net .


The trilateral weighted sparse coding (TWSC) #REFR accomplishes real image denoising by employing image priors.

As one of the most significant research areas of low-level visual tasks, image denoising aims to restore clean images from noisy ones.

During the past decades, many researchers have presented a number of denoising methods.

Before the wide application of deep neural networks (DNNs), filtering techniques and sparse learning are widely used denoising methods.

For instance, in the NLM #OTHEREFR , the weighted average of all pixels within the search window in an image is applied to achieve noise removal.

The BM3D #OTHEREFR improves the sparse representation by collaborative alteration.

The weighted nuclear norm minimization (WNNM) #OTHEREFR and the multi-channel WNNM (MCWNNM) for color images #OTHEREFR employ the low rank approach and prior knowledge to enhance denoising performance.

These denoising methods can obtain favorable denoising performance, however most of them have to contain a complex and timeconsuming optimization algorithm.

Meanwhile, many manually adjusted parameters are also usually required for these models to perform well, this may lead to uncertainty of their denoising performance.

Therefore these models can hardly be applied in practical denoising scenes.

From the early successful deep neural networks (DNNs) based image denoising model DnCNN #OTHEREFR to the present, the DNNs based denoising models have received much attention due to their superior denoising effect.

Dcanet: Dual Convolutional Neural Network With Attention For Image Blind Denoising

Abstract: Noise removal of images is an essential preprocessing procedure for many computer vision tasks. Currently, many denoising models based on deep neural networks can perform well in removing the noise with known distributions (i.e. the additive Gaussian white noise). However eliminating real noise is still a very challenging task, since real-world noise often does not simply follow one single type of distribution, and the noise may spatially vary. In this paper, we present a new dual convolutional neural network (CNN) with attention for image blind denoising, named as the DCANet. To the best of our knowledge, the proposed DCANet is the first work that integrates both the dual CNN and attention mechanism for image denoising. The DCANet is composed of a noise estimation network, a spatial and channel attention module (SCAM), and a CNN with a dual structure. The noise estimation network is utilized to estimate the spatial distribution and the noise level in an image. The noisy image and its estimated noise are combined as the input of the SCAM, and a dual CNN contains two different branches is designed to learn the complementary features to obtain the denoised image. The experimental results have verified that the proposed DCANet can suppress both synthetic and real noise effectively.

A Trilateral Weighted Sparse Coding Scheme For Real-World Image Denoising

Abstract: Most of existing image denoising methods assume the corrupted noise to be additive white Gaussian noise (AWGN). However, the realistic noise in real-world noisy images is much more complex than AWGN, and is hard to be modelled by simple analytical distributions. As a result, many state-of-the-art denoising methods in literature become much less effective when applied to real-world noisy images captured by CCD or CMOS cameras. In this paper, we develop a trilateral weighted sparse coding (TWSC) scheme for robust real-world image denoising. Specifically, we introduce three weight matrices into the data and regularisation terms of the sparse coding framework to characterise the statistics of realistic noise and image priors. TWSC can be reformulated as a linear equality-constrained problem and can be solved by the alternating direction method of multipliers. The existence and uniqueness of the solution and convergence of the proposed algorithm are analysed. Extensive experiments demonstrate that the proposed TWSC scheme outperforms state-of-the-art denoising methods on removing realistic noise.


Among the four word embeddings models we tried, the best performing is FastText, confirming the results of #REFR .

The results of the Spearman's ρ correlation in the STS benchmark of our SFBoW are reported in the last three rows of Table 2 .

The reported values belong to the FSBoW configurations that achieved a best score, among the variants we considered for the experiments, in at least one task; if interested in the complete experimental results reporting all the SFBoW configurations, please refer to Appendix A.

Concerning the choice of the universe matrix, instead, the best scores are achieved either with Identity matrix or with PCA rotation matrix, highlighting how the features described by the word embeddings provide a better description of the semantic content of sentences.

About the choice of the universe matrix, clustering provided poor results, so Table 2 reports only the scores from Identity matrix and PCA.

Density based clustering turned out to give meaningless results, for this reason its analysis is omitted.

k-Means clustering, instead, gave more promising results, but still f Universe matrix is the identity matrix. g Universe matrix is the PCA projection matrix. h Universe matrix is built from the English vocabulary.

i Universe matrix is built from the top 100 000 most frequent words.

Static Fuzzy Bag-Of-Words: A Lightweight Sentence Embedding Algorithm

Abstract: The introduction of embedding techniques has pushed forward significantly the Natural Language Processing field. Many of the proposed solutions have been presented for word-level encoding; anyhow, in the last years, new mechanism to treat information at an higher level of aggregation, like at sentence-and document-level, have emerged. With this work we address specifically the sentence embeddings problem, presenting the Static Fuzzy Bag-of-Word model. Our model is a refinement of the Fuzzy Bag-of-Words approach, providing sentence embeddings with a predefined dimension. SFBoW provides competitive performances in Semantic Textual Similarity benchmarks, while requiring low computational resources.

Don'T Settle For Average, Go For The Max: Fuzzy Sets And Max-Pooled Word Vectors

Abstract: Recent literature suggests that averaged word vectors followed by simple post-processing outperform many deep learning methods on semantic textual similarity tasks. Furthermore, when averaged word vectors are trained supervised on large corpora of paraphrases, they achieve state-of-the-art results on standard STS benchmarks. Inspired by these insights, we push the limits of word embeddings even further. We propose a novel fuzzy bag-of-words (FBoW) representation for text that contains all the words in the vocabulary simultaneously but with different degrees of membership, which are derived from similarities between word vectors. We show that max-pooled word vectors are only a special case of fuzzy BoW and should be compared via fuzzy Jaccard index rather than cosine similarity. Finally, we propose DynaMax, a completely unsupervised and non-parametric similarity measure that dynamically extracts and max-pools good features depending on the sentence pair. This method is both efficient and easy to implement, yet outperforms current baselines on STS tasks by a large margin and is even competitive with supervised word vectors trained to directly optimise cosine similarity.


In addition to the dependency of these NLP-powered tools on the availability and reliability of training data, this puts the NLP4RE research domain on the forefront of the open science challenge #REFR .

One popular approach to this is the proposal of quality factors.

Requirements quality publications often formulate one or more quality factors-e.g., the use of coordination ambiguity leading to divergent interpretations #OTHEREFR annotate instances of that quality factor in a data set, and finally present an implementation (i.e., an algorithm or full-fledged tool) to detect these instances automatically.

These artifacts-both data sets and implementations-represent essential contributions facilitating empirical research and technology transfer.

While the (annotated) data sets are the main driver for developing new and improving existing implementations for quality factor detection, implementations are the tools to be deployed in industry for actual integration and improvement of the software engineering process.

The NLP4RE research domain, which applies natural language processing (NLP) techniques to RE #OTHEREFR and constitutes a large part of the contributions to the requirements quality literature [1] , is particularly focused on said delivery and improvement of tools.

The NLP4RE community is therefore particularly aware of its dependency on the availability of artifacts #OTHEREFR .

However, recent systematic studies revealed that a significant amount of these artifacts are not available 2 anymore or have never been #OTHEREFR 1, #OTHEREFR .

Table 1 reports the availability status of 57 data sets (D) and 36 implementations (I) extracted from the 57 primary studies of our previously-published literature review on requirements quality factors #OTHEREFR .

Let'S Stop Building At The Feet Of Giants: Recovering Unavailable Requirements Quality Artifacts

Abstract: Requirements quality literature abounds with publications presenting artifacts, such as data sets and tools. However, recent systematic studies show that more than 80% of these artifacts have become unavailable or were never made public, limiting reproducibility and reusability. In this work, we report on an attempt to recover those artifacts. To that end, we requested corresponding authors of unavailable artifacts to recover and disclose them according to open science principles. Our results, based on 19 answers from 35 authors (54% response rate), include an assessment of the availability of requirements quality artifacts and a breakdown of authors' reasons for their continued unavailability. Overall, we improved the availability of seven data sets and seven implementations.

Natural Language Processing (Nlp) For Requirements Engineering: A Systematic Mapping Study

Abstract: Natural language processing supported requirements engineering is an area of research and development that seeks to apply NLP techniques, tools and resources to a variety of requirements documents or artifacts to support a range of linguistic analysis tasks performed at various RE phases. Such tasks include detecting language issues, identifying key domain concepts and establishing traceability links between requirements. This article surveys the landscape of NLP4RE research to understand the state of the art and identify open problems. The systematic mapping study approach is used to conduct this survey, which identified 404 relevant primary studies and reviewed them according to five research questions, cutting across five aspects of NLP4RE research, concerning the state of the literature, the state of empirical research, the research focus, the state of the practice, and the NLP technologies used. Results: 1) NLP4RE is an active and thriving research area in RE that has amassed a large number of publications and attracted widespread attention from diverse communities; 2) most NLP4RE studies are solution proposals having only been evaluated using a laboratory experiment or an example application; 3) most studies have focused on the analysis phase, with detection as their central linguistic analysis task and requirements specification as their commonly processed document type; 4) 130 new tools have been proposed to support a range of linguistic analysis tasks, but there is little evidence of adoption in the long term, although some industrial applications have been published; 5) 140 NLP techniques, 66 NLP tools and 25 NLP resources are extracted from the selected studies.


Thus, the overall performance of the English mapping is 99.89%, which compares favorably with more complex mapping strategies like #REFR .

We evaluate the performance of our algorithm using the values above, and obtain almost perfect performance results:

precision = tp tp + f p = 0.9996 (1) recall = tp tp + f n = 0.9983 (2) f 1 = 2 * precision * recall precision + recall = 0.9989 (3)

Comparing the lost English synsets between the two types of synset identifiers (offsets vs.

ILIs), we found that 143 were lost using both types, while 62 were only lost with offsets (always due to satellite adjectives becoming standard adjectives), and 89 were only lost with CILI 1.0.

The respective additions of these losses yield the total loss reported for English in table 1 (205 with offsets vs. 232 with the ILI).

Mapping Wordnets On The Fly With Permanent Sense Keys

Abstract: Most of the major databases on the semantic web have links to Princeton WordNet (PWN) synonym set (synset) identifiers, which differ for each PWN release, and are thus incompatible between versions. On the other hand, both PWN and the more recent Open English Wordnet (OEWN) provide permanent word sense identifiers (the sense keys), which can solve this interoperability problem. We present an algorithm that runs in linear time, to automatically derive a synset mapping between any pair of Wordnet versions that use PWN sense keys. This allows to update old WordNet links, and seamlessly interoperate with newer English Wordnet versions for which no prior mapping exists. By applying the proposed algorithm on the fly, at load time, we combine the Open Multilingual Wordnet (OMW 1.4, which uses old PWN 3.0 identifiers) with OEWN Edition 2021, and obtain almost perfect precision and recall. We compare the results of our approach using respectively synset offsets, versus the Collaborative InterLingual Index (CILI version 1.0) as synset identifiers, and find that the synset offsets perform better than CILI 1.0 in all cases, except a few ties.

Mapping Wordnets Using Structural Information

Abstract: We present a robust approach for linking already existing lexical/semantic hierarchies. We used a constraint satisfaction algorithm (relaxation labeling) to select - among a set of candidates- the node in a target taxonomy that bests matches each node in a source taxonomy. In particular, we use it to map the nominal part of WordNet 1.5 onto WordNet 1.6, with a very high precision and a very low remaining ambiguity.


To obtain human skeletal data in the scene we compared the output of the Convolutional Pose Moachine (CPM) #REFR model with the kinect skeletal data.

Table 4 summarizes the characteristics of the datasets employed in this study.

Moreover, the affordance labels present in the LOAD dataset are indicated in Figure 8 , whereas the labels for both the CAD-120 and Watch-n-Patch datasets are: 'can-contain', 'containable', 'can-support', 'supportable', 'drinkable', and 'holdable'.

supportable can-support sittable  holdable  pullable  can-cover  rollable  pushable  carriable  kickable  coverable  can-contain containable drinkable  bouncable We used 80% of the CAD-120 dataset to determine the method's parameters and train the graph2vec network, resulting to 18,072 AGraphlets, and then evaluated the proposed approach on the remaining 20% unseen videos (24 videos), which comprise of 5,682 AGraphlets.

Experiments on the Watch-n-Patch and LOAD datasets were conducted on 24 and 15 handpicked videos, or 6,900 and 1,212 AGraphlets respectively, where the predictions of the object detector, after visual inspection, were sufficient for capturing interacting objects.

For defining the parameters of the algorithm, we exploited hand-picked videos from the training set for the CAD-120 dataset, and for the LOAD and Watch-n-Patch datasets we hand-picked videos different from the ones used for testing the proposed method.

Through an empirical study we found that the CPM predictions are more accurate, especially when human-joint occlusion occurs or the human agent stands on their side.

Object locations and depth information are provided from the predicted objects' masks.

We employ the state-of-the-art Mask R-CNN framework #OTHEREFR ) trained on the COCO dataset #OTHEREFR , since the training data distribution is similar to the data distribution present in the datasets we use in this work, and it is a generic dataset with more object classes than those included in the target task.

The box enclosing the object's mask corresponds to the object's bounding box.

This implementation is based on Mask R-CNN predictions, however the proposed method is not object-specific and any class-agnostic proposal method can be used.

Object-Agnostic Affordance Categorization Via Unsupervised Learning Of Graph Embeddings

Abstract: Acquiring knowledge about object interactions and affordances can facilitate scene understanding and human-robot collaboration tasks. As humans tend to use objects in many different ways depending on the scene and the objects' availability, learning object affordances in everyday-life scenarios is a challenging task, particularly in the presence of an open set of interactions and objects. We address the problem of affordance categorization for class-agnostic objects with an open set of interactions; we achieve this by learning similarities between object interactions in an unsupervised way and thus inducing clusters of object affordances. A novel depth-informed qualitative spatial representation is proposed for the construction of Activity Graphs (AGs), which abstract from the continuous representation of spatio-temporal interactions in RGB-D videos. These AGs are clustered to obtain groups of objects with similar affordances. Our experiments in a real-world scenario demonstrate that our method learns to create object affordance clusters with a high V-measure even in cluttered scenes. The proposed approach handles object occlusions by capturing effectively possible interactions and without imposing any object or scene constraints.

Convolutional Pose Machines

Abstract: Pose Machines provide a sequential prediction framework for learning rich implicit spatial models. In this work we show a systematic design for how convolutional networks can be incorporated into the pose machine framework for learning image features and image-dependent spatial models for the task of pose estimation. The contribution of this paper is to implicitly model long-range dependencies between variables in structured prediction tasks such as articulated pose estimation. We achieve this by designing a sequential architecture composed of convolutional networks that directly operate on belief maps from previous stages, producing increasingly refined estimates for part locations, without the need for explicit graphical model-style inference. Our approach addresses the characteristic difficulty of vanishing gradients during training by providing a natural learning objective function that enforces intermediate supervision, thereby replenishing back-propagated gradients and conditioning the learning procedure. We demonstrate state-of-the-art performance and outperform competing methods on standard benchmarks including the MPII, LSP, and FLIC datasets.


It is substantially similar to that of the general result given in #REFR (see Theorem 1) except that the result here relates to formulas and not to theories.

A revision operator • satisfies the AGM postulates if and only if there exists a FA that maps each formula ϕ to a binary relation ϕ such that for every formula ψ

Min(Mod(ψ), ϕ ) = {M ∈ Mod(ψ) | ∀M ′ ∈ Mod(ψ), M ′ ≺ ϕ M}.

The proof of Proposition 16 is given in Appendix.

This then required redefining a particular FA in the proof.

The FA used in #OTHEREFR to prove this representation result was an adaptation of the FA proposed in the original paper #OTHEREFR , but the latter was not adaptable within the framework of modal logic 6 .

Morpho-Logic From A Topos Perspective: Application To Symbolic Ai

Abstract: Modal logics have proved useful for many reasoning tasks in symbolic artificial intelligence (AI), such as belief revision, spatial reasoning, among others. On the other hand, mathematical morphology (MM) is a theory for non-linear analysis of structures, that was widely developed and applied in image analysis. Its mathematical bases rely on algebra, complete lattices, topology. Strong links have been established between MM and mathematical logics, mostly modal logics. In this paper, we propose to further develop and generalize this link between mathematical morphology and modal logic from a topos perspective, i.e. categorial structures generalizing space, and connecting logics, sets and topology. Furthermore, we rely on the internal language and logic of topos. We define structuring elements, dilations and erosions as morphisms. Then we introduce the notion of structuring neighborhoods, and show that the dilations and erosions based on them lead to a constructive modal logic, for which a sound and complete proof system is proposed. We then show that the modal logic thus defined (called morpho-logic here), is well adapted to define concrete and efficient operators for revision, merging, and abduction of new knowledge, or even spatial reasoning.

Belief Revision, Minimal Change And Relaxation: A General Framework Based On Satisfaction Systems, And Applications To Description Logics

Abstract: Belief revision of knowledge bases represented by a set of sentences in a given logic has been extensively studied but for specific logics, mainly propositional, and also recently Horn and description logics. Here, we propose to generalize this operation from a model-theoretic point of view, by defining revision in the abstract model theory of satisfaction systems. In this framework, we generalize to any satisfaction system the characterization of the AGM postulates given by Katsuno and Mendelzon for propositional logic in terms of minimal change among interpretations. In this generalization, the constraint on syntax independence is partially relaxed. Moreover, we study how to define revision, satisfying these weakened AGM postulates, from relaxation notions that have been first introduced in description logics to define dissimilarity measures between concepts, and the consequence of which is to relax the set of models of the old belief until it becomes consistent with the new pieces of knowledge. We show how the proposed general framework can be instantiated in different logics such as propositional, first-order, description and Horn logics. In particular for description logics, we introduce several concrete relaxation operators tailored for the description logic ALC and its fragments EL and ELU , discuss their properties and provide some illustrative examples.


The algorithm used to achieve this was UNet #REFR , it uses Double Convolutional layers to identify and extract features from the input image and uses skip connects to reuse these features in a related layer.

In this case, the pattern of interest is the rot present in an image, while identifying rot is a subjective matter it is entirely possible to map this as an input feature to a model and have its representations capture.

This model would be able to capture image segments on related images.

To achieve this, the model typically uses a combination of Double Convolutional Neural Networks with a structure called skip-connections which skip some of the connections in a neural network and feeds the output of one layer as input to the other layers.

Skip-connections greatly reduce the complexity of loss surfaces, making it easier for optimizers to reduce loss while ensuring that feature representations are reused #OTHEREFR .

The images for a sample image and prediction are shown below (areas in yellow are rotten areas of the meat as identified by the model).

The idea is that each feature set captured in a layer is captured in a layer connected by a skip connection and passed to the next layer to compute the representation segment.

Since this task outputs a set of image patterns the ideal outcome would be identifying the quality of the outputs in terms of the intersection and the overlap resulting from the predictions and the image masks.

The loss functions capable of representing this effort are Dice loss and Jaccard loss which broadly look at the ratio of the intersection to the union, so concretely both would have a measure of how well the model can segment the patterns of interest from a given input image.

The extracted image segmented predictions were passed as input features to a DenseNet model and predictions were output based on the segments captured.

The segment of interest in this case is the rot present in the image and this was one-hot encoded when it was passed to the DenseNet model.

Meat Freshness Prediction

Abstract: In most retail stores, the number of days since initial processing is used as a proxy for estimating the freshness of perishable foods or freshness is assessed manually by an employee. While the former method can lead to wastage, as some fresh foods might get disposed after a fixed number of days, the latter can be time-consuming, expensive and impractical at scale. This project aims to propose a Machine Learning (ML) based approach that evaluates freshness of food based on live data. For the current scope, it only considers meat as a the subject of analysis and attempts to classify pieces of meat as fresh, half-fresh or spoiled. Finally the model achieved an accuracy of above 90% and relatively high performance in terms of the cost of misclassification. It is expected that the technology will contribute to the optimization of the client's business operation, reducing the risk of selling defective or rotten products that can entail serious monetary, non-monetary and health-based consequences while also achieving higher corporate value as a sustainable company by reducing food wastage through timely sales and disposal. 1. Business Problem Assessing the freshness of perishable food is a significant operational challenge for retailers, as it is time-consuming, and can affect their business performance as well as reputation if a wrong judgment is made. In most retail stores, the number of days since initial processing is used as a proxy for freshness. Regardless of actual freshness, products are judged to be fresh if fewer days have passed and stale if more days have passed. Products that have passed many days since their initial processing are discounted, and if they are still not purchased, they are ultimately disposed


By distinguishing specific speakers, DialogueRNN #REFR modeled emotions dynamically based on the current speaker, contextual content, and emotional state. Zhong et al.

Distinct from traditional emotion recognition which treats emotion as a static state, ERC takes full consideration of emotion to be dynamic and flow between speaker interactions. Hazarika et al.

#OTHEREFR proposed a LSTM-based model to enable current utterance to capture contextual information in historical conversations.

CMN #OTHEREFR employed a skip attention mechanism to merge contextual information in a historical conversation. Jiao et al.

#OTHEREFR proposed a hierarchical GRU to address the difficulty of capturing longdistance contextual information effectively.

#OTHEREFR proposed Knowledge-Enriched Transformer, which dynamically exploited external commonsense knowledge through hierarchical self-attention and context-aware graph attention.

By building directed graphical structures over the input utterance sequences with speaker information, DialogueGCN #OTHEREFR applied graph convolution network to construct inter-and intra-dependencies among distant utterances.

COSMIC #OTHEREFR combined different commonsense knowledge and learned the interaction between the interlocutors in the dialogue.

DialogXL #OTHEREFR modified the memory block in XLNet #OTHEREFR to store longer historical contexts and conversation-aware self-attention to handle multi-party structures. Wang et al.

#OTHEREFR proposed a relational graph attention network to encode the tree structure for sentiment prediction.

Research Paper . Emotionic: Emotional Inertia And Contagion-Driven Dependency Modelling For Emotion Recognition In Conversation

Abstract: Emotion Recognition in Conversation (ERC) has attracted growing attention in recent years as a result of the advancement and implementation of human-computer interface technologies. However, previous approaches to modeling global and local context dependencies lost the diversity of dependency information and do not take the context dependency into account at the classification level. In this paper, we propose a novel approach to dependency modeling driven by Emotional Inertia and Contagion (EmotionIC) for conversational emotion recognition at the feature extraction and classification levels. At the feature extraction level, our designed Identity Masked Multi-head Attention (IM-MHA) captures the identity-based long-distant context in the dialogue to contain the diverse influence of different participants and construct the global emotional atmosphere, while the devised Dialogue-based Gate Recurrent Unit (DialogGRU) that aggregates the emotional tendencies of dyadic dialogue is applied to refine the contextual features with interand intra-speaker dependencies. At the classification level, by introducing skip connections in Conditional Random Field (CRF), we elaborate the Skip-chain CRF (SkipCRF) to capture the high-order dependencies within and between speakers, and to emulate the emotional flow of distant participants. Experimental results show that our method can significantly outperform the state-of-the-art models on four benchmark datasets. The ablation studies confirm that our modules can effectively model emotional inertia and contagion.

Dialoguernn: An Attentive Rnn For Emotion Detection In Conversations

Abstract: Emotion detection in conversations is a necessary step for a number of applications, including opinion mining over chat history, social media threads, debates, argumentation mining, understanding consumer feedback in live conversations, and so on. Currently systems do not treat the parties in the conversation individually by adapting to the speaker of each utterance. In this paper, we describe a new method based on recurrent neural networks that keeps track of the individual party states throughout the conversation and uses this information for emotion classification. Our model outperforms the state-of-the-art by a significant margin on two different datasets.


Comparisons With State-Of-The-Art Methods

And with the help of powerful SR backbone (RRDB-Net #REFR with 16.70M parameters), their results are quite remarkable.

We also conduct experiments on BasicVSR #OTHEREFR and PDM #OTHEREFR as two representative methods for noneblind VSR method and blind SISR with implicit degradation modeling.

Apart from AnimeSR #OTHEREFR , for which we report the result in the original paper, all the others are general SR methods for open-domain images or videos.

For fair comparisons, we fine-tune their officially released models on animation dataset AVC-Train. Quantitative comparison. As shown in Tab.

1, we evaluate all the methods on AVC-ReaLQ #OTHEREFR for quantitative comparisons.

Among them, Real-ESRGAN #OTHEREFR and BSR-GAN #OTHEREFR expand the former explicit degradation models by introducing high-order degradations and random shuffling respectively, which greatly improve the synthetic ability.

Nonetheless, ignoring the intrinsic characteristics of animation data limits their performance when applied to animation domain.

Although specialized for animation videos, AnimeSR #OTHEREFR only utilizes a small number of real data (three human-annotated animation videos), which hinders the performance of VSR model in real scenarios.

Different from them, our VQD-SR considers the intrinsic characteristics of animation videos and leverages the enormous degradation priors contained in rich-content real animation videos.

VQD-SR also adopts the HR enhancement strategy for more effective SR supervision.

Due to these advantages, VQD-SR achieves a result of 0.4096 and significantly outperforms the SOTA animation VSR model #OTHEREFR by 0.0264 in MANIQA on AVC-RealLQ #OTHEREFR .

Learning Data-Driven Vector-Quantized Degradation Model For Animation Video Super-Resolution

Abstract: Existing real-world video super-resolution (VSR) methods focus on designing a general degradation pipeline for open-domain videos while ignoring data intrinsic characteristics which strongly limit their performance when applying to some specific domains (e.g. animation videos). In this paper, we thoroughly explore the characteristics of animation videos and leverage the rich priors in real-world animation data for a more practical animation VSR model. In particular, we propose a multi-scale Vector-Quantized Degradation model for animation video Super-Resolution (VQD-SR) to decompose the local details from global structures and transfer the degradation priors in real-world animation videos to a learned vector-quantized codebook for degradation modeling. A rich-content Real Animation Low-quality (RAL) video dataset is collected for extracting the priors. We further propose a data enhancement strategy for high-resolution (HR) training videos based on our observation that existing HR videos are mostly collected from the Web which contains conspicuous compression artifacts. The proposed strategy is valid to lift the upper bound of animation VSR performance, regardless of the specific VSR model. Experimental results demonstrate the superiority of the proposed VQD-SR over state-of-the-art methods, through extensive quantitative and qualitative evaluations of the latest animation video super-resolution benchmark.

Esrgan: Enhanced Super-Resolution Generative Adversarial Networks

Abstract: The Super-Resolution Generative Adversarial Network (SRGAN) is a seminal work that is capable of generating realistic textures during single image super-resolution. However, the hallucinated details are often accompanied with unpleasant artifacts. To further enhance the visual quality, we thoroughly study three key components of SRGAN - network architecture, adversarial loss and perceptual loss, and improve each of them to derive an Enhanced SRGAN (ESRGAN). In particular, we introduce the Residual-in-Residual Dense Block (RRDB) without batch normalization as the basic network building unit. Moreover, we borrow the idea from relativistic GAN to let the discriminator predict relative realness instead of the absolute value. Finally, we improve the perceptual loss by using the features before activation, which could provide stronger supervision for brightness consistency and texture recovery. Benefiting from these improvements, the proposed ESRGAN achieves consistently better visual quality with more realistic and natural textures than SRGAN and won the first place in the PIRM2018-SR Challenge. The code is available at this https URL .


Review Of Brain Glioma Segmentation Methods

To avoid over-fitting problems in 3D voxel-level segmentation on limited training datasets, Myronenko #REFR proposed a 3D CNN with an additional variational autoencoder to regularise the decoder by reconstructing the input image.

#OTHEREFR ensembled three 2D CNNs on three orthogonal 2D patches.

To fully make use of 3D contextual information, recent works applied 3D convolutional kernels on original volume data. Kamnitsas et al.

#OTHEREFR proposed two pathway 3D CNN followed with dense CRF called DeepMedic for brain tumor segmentation.

Authors of #OTHEREFR further extended the work by using model ensembling #OTHEREFR .

The proposed system EMMA ensembled models from FCN, U-Net and DeepMedic for processing 3D patches.

The architecture built in #OTHEREFR is further developed in various recent works. Su et al.

#OTHEREFR extends the architecture built in #OTHEREFR into two sub-networks to fuse the information learned from different modalities. Jiang et al.

#OTHEREFR proposed two-stage networks where each stage adopts a similar network in #OTHEREFR .

The first stage network generates a coarse result and the second stage network refines the segmentation result.

The final result in #OTHEREFR reaches state-of-the-art by ensemble 12 model instances, which requires huge computational resources.

Medical Image Analysis Using Deep Relational Learning

Abstract: School of Informatics Master of Philosophy Medical Image Analysis using Deep Relational Learning by Zhihua LIU Benefited from deep learning techniques, remarkable progress has been made within the medical image analysis area in recent years. However, it is very challenging to fully utilize the relational information (the relationship between tissues or organs or images) within the deep neural network architecture. Thus in this thesis, we propose two novel solutions to this problem called implicit and explicit deep relational learning. We generalize these two paradigms of deep relational learning into different solutions and evaluate them on various medical image analysis tasks. Automated segmentation of brain glioma in 3D magnetic resonance imaging plays an active role in glioma diagnosis, progression monitoring and surgery planning. In this work, we propose a novel Context-Aware Network that effectively models implicit relation information between features to perform accurate 3D glioma segmentation. We evaluate our proposed method on publicly accessible brain tumor segmentation datasets BRATS2017 and BRATS2018 against several state-of-the-art approaches using different segmentation metrics. The experimental results show that the proposed algorithm has better or competitive performance, compared to the standard approaches. Subsequently, we propose a new hierarchical homography estimation network to achieve accurate medical image mosaicing by learning the explicit spatial relationship between adjacent frames. We use the UCL Fetoscopy Placenta dataset to conduct experiments and our hierarchical homography estimation network outperforms the other state-of-the-art mosaicing methods while generating robust and meaningful mosaicing results on unseen frames. First of all, I want to thank my father Mr. Yuhua Du, my mother Mrs. Zengxia Xiao and my fiancee Miss. Xinyu Wang. Thank them for their support to me and the family. I cannot repay the support from my family, and it is also my biggest motivation to continue scientific research. Secondly, I want to thank my first supervisor, Prof. Huiyu Zhou. Prof. Huiyu Zhou has a rigorous attitude towards science. He also dedicates himself to his work. He is both a good teacher and a good friend. I am grateful to him for his continuous criticism and teaching, which have benefited me for life. At the same time, I would also like to thank my second supervisor, Prof. Yudong Zhang, for his professional guidance on my academic work. Also, I would like to thank all colleagues of Biomedical Image Processing Lab (BIPL), especially Mr.

3D Mri Brain Tumor Segmentation Using Autoencoder Regularization

Abstract: Automated segmentation of brain tumors from 3D magnetic resonance images (MRIs) is necessary for the diagnosis, monitoring, and treatment planning of the disease. Manual delineation practices require anatomical knowledge, are expensive, time consuming and can be inaccurate due to human error. Here, we describe a semantic segmentation network for tumor subregion segmentation from 3D MRIs based on encoder-decoder architecture. Due to a limited training dataset size, a variational auto-encoder branch is added to reconstruct the input image itself in order to regularize the shared decoder and impose additional constraints on its layers. The current approach won 1st place in the BraTS 2018 challenge.

3D CNN
additional variational autoencoder


To ensure that RG f is the Reeb graph of a Morse function, we first eliminate degenerate critical nodes in RG f by breaking them into nondegenerate critical nodes #REFR .

ExDg 0 (RG f ) encodes the range of f and ExDg 1 (RG f ) captures the 1-cycles or loops in RG f .

To compute the persistence diagrams of RG f , we require RG f to be the Reeb graph of a Morse function. If f is a Morse function, i.e.

all its critical points are non-degenerate and are at different levels, then the critical nodes of RG f have distinct values off and belong to one of the following five-types: (i) a minimum (with down-degree = 0, up-degree = 1), (ii) a maximum (with up-degree = 0, down-degree = 1), (iii) a downfork (with down-degree = 2, up-degree = 1) and (iv) a up-fork (with up-degree = 2, down-degree = 1).

A regular node has up-degree = 1 and down-degree = 1.

A down-fork (similarly, up-fork) node is called an essential down-fork node when it contributes to a loop (cycle) of the Reeb graph. Otherwise it is called an ordinary down-fork node.

After eliminating degenerate critical nodes, we ensure that the critical nodes off are at different levels.

If two critical nodes are at the same level, then the value of one of the nodes is increased/decreased by a small value .

After removing degenerate critical nodes and ensuring that critical nodes are at different levels, RG f becomes the Reeb graph of a Morse function.

The points in Dg 0 (f ) are computed by pairing ordinary down-forks with minima, ordinary up-forks with maxima and the global minimum with global maximum.

Let u be an ordinary down-fork of RG f .

A Topological Distance Measure Between Multi-Fields For Classification And Analysis Of Shapes And Data

Abstract: Distance measures play an important role in shape classification and data analysis problems. Topological distances based on Reeb graphs and persistence diagrams have been employed to obtain effective algorithms in shape matching and scalar data analysis. In the current paper, we propose an improved distance measure between two multi-fields by computing a multidimensional Reeb graph (MDRG) each of which captures the topology of a multi-field through a hierarchy of Reeb graphs in different dimensions. A hierarchy of persistence diagrams is then constructed by computing a persistence diagram corresponding to each Reeb graph of the MDRG. Based on this representation, we propose a novel distance measure between two MDRGs by extending the bottleneck distance between two Reeb graphs. We show that the proposed measure satisfies the pseudo-metric and stability properties. We examine the effectiveness of the proposed multi-field topologybased measure on two different applications: (1) shape classification and (2) detection of topological features in a time-varying multi-field data. In the shape classification problem, the performance of the proposed measure is compared with the well-known topology-based measures in shape matching. In the second application, we consider a time-varying volumetric multi-field data from the field of computational chemistry where the goal is to detect the site of stable bond formation between Pt and CO molecules. We demonstrate the ability of the proposed distance in classifying each of the sites as occurring before and after the bond stabilization.

Propagate And Pair: A Single-Pass Approach To Critical Point Pairing In Reeb Graphs

Abstract: With the popularization of Topological Data Analysis, the Reeb graph has found new applications as a summarization technique in the analysis and visualization of large and complex data, whose usefulness extends beyond just the graph itself. Pairing critical points enables forming topological fingerprints, known as persistence diagrams, that provides insights into the structure and noise in data. Although the body of work addressing the efficient calculation of Reeb graphs is large, the literature on pairing is limited. In this paper, we discuss two algorithmic approaches for pairing critical points in Reeb graphs, first a multipass approach, followed by a new single-pass algorithm, called Propagate and Pair.