The PoE method #REFR composited representations of entities by concatenating all modality features, which could not capture the potential interactions among heterogeneous modalities and therefore limited the performance of MMEA.

Recently, there is a growing trend to incorporate multi-modal information into KGs, i.e., Multi-Modal Knowledge Graphs (MMKGs), which support various cross-modal tasks, e.g., recommendation systems #OTHEREFR and question answering systems #OTHEREFR . However, MMKGs often suffer from low coverage and incompleteness.

To improve the coverage of these MMKGs, a viable approach termed as multi-modal entity alignment (MMEA) is proposed to identify the equivalent entity pairs (i.e., alignment seeds) in different MMKGs, by integrating the attribute information of text and image.

In this way, MMKGs can obtain useful knowledge from other KG.

Although the rich attributes and neighboring entities in MMKGs provide valuable pieces of evidence for MMEA #OTHEREFR , the inevitable heterogeneity of MMKGs makes it difficult to learn and fuse knowledge representations from distinct modalities.

A series of effective methods have been developed to conquer these challenges, and the detailed description is in Appendix A.

Later works #OTHEREFR designed multi-modal fusion modules to properly integrate attributes and entities, in order to better predict alignments according to aggregated embeddings.

All of these methods would learn entity representations by harnessing their whole associated attributes and neighboring entities.

Nevertheless, they ignore the contextual gaps between entity pairs and in turn constrain the effectiveness of the entity alignment.

The contextual gap, which means entities may associate with different numbers of attributes or even lack some modalities, is inevitable due to information redundancy or absence.

Such inconsistencies between the equivalent entity pairs make alignments error-prone.

Attribute-Consistent Knowledge Graph Representation Learning For Multi-Modal Entity Alignment

Abstract: The multi-modal entity alignment (MMEA) aims to find all equivalent entity pairs between multi-modal knowledge graphs (MMKGs). Rich attributes and neighboring entities are valuable for the alignment task, but existing works ignore contextual gap problems that the aligned entities have different numbers of attributes on specific modality when learning entity representations. In this paper, we propose a novel attribute-consistent knowledge graph representation learning framework for MMEA (ACK-MMEA) to compensate the contextual gaps through incorporating consistent alignment knowledge. Attribute-consistent KGs (ACKGs) are first constructed via multi-modal attribute uniformization with merge and generate operators so that each entity has one and only one uniform feature in each modality. The ACKGs are then fed into a relation-aware graph neural network with random dropouts, to obtain aggregated relation representations and robust entity representations. In order to evaluate the ACK-MMEA facilitated for entity alignment, we specially design a joint alignment loss for both entity and attribute evaluation. Extensive experiments conducted on two benchmark datasets show that our approach achieves excellent performance compared to its competitors.

Mmkg: Multi-Modal Knowledge Graphs

Abstract: We present Mmkg, a collection of three knowledge graphs that contain both numerical features and (links to) images for all entities as well as entity alignments between pairs of KGs. Therefore, multi-relational link prediction and entity matching communities can benefit from this resource. We believe this data set has the potential to facilitate the development of novel multi-modal learning approaches for knowledge graphs. We validate the utility of Mmkg in the \(\mathtt {sameAs}\) link prediction task with an extensive set of experiments. These experiments show that the task at hand benefits from learning of multiple feature types.


The maximal eigenvalue, λ 1 , can be interpreted as measuring the sharpness of the loss landscape #REFR , and the condition number λ 1 /λ d has significant implications for convergence of gradient descent optimization in deep neural networks (Gilmer et al., 2022) .

The Hessian of a network f at parameters θ with respect to some loss (θ) is a matrix defined as

where d = |θ| is the number of parameters.

Of particular relevance to optimization is the eigenspectrum of the Hessian

We will also take interest in the covariance structure of the gradients of different data points in the input distribution, a property relevant to both optimization and generalization #OTHEREFR .

We will estimate this covariance structure by sampling k training points x 1 , . . .

, x k , and computing the matrix C k ∈ R k×k defined entrywise as

If the off-diagonal entries of C k contain many negative values, this indicates interference between inputs, wherein the network cannot reduce its loss on one region without increasing its loss on another.

Understanding Plasticity In Neural Networks

Abstract: Plasticity, the ability of a neural network to quickly change its predictions in response to new information, is essential for the adaptability and robustness of deep reinforcement learning systems. Deep neural networks are known to lose plasticity over the course of training even in relatively simple learning problems, but the mechanisms driving this phenomenon are still poorly understood. This paper conducts a systematic empirical analysis into plasticity loss, with the goal of understanding the phenomenon mechanistically in order to guide the future development of targeted solutions. We find that loss of plasticity is deeply connected to changes in the curvature of the loss landscape, but that it typically occurs in the absence of saturated units or divergent gradient norms. Based on this insight, we identify a number of parameterization and optimization design choices which enable networks to better preserve plasticity over the course of training. We validate the utility of these findings in larger-scale learning problems by applying the best-performing intervention, layer normalization, to a deep RL agent trained on the Arcade Learning Environment.

Sharp Minima Can Generalize For Deep Nets

Abstract: Despite their overwhelming capacity to overfit, deep learning architectures tend to generalize relatively well to unseen data, allowing them to be deployed in practice. However, explaining why this is the case is still an open area of research. One standing hypothesis that is gaining popularity, e.g. Hochreiter & Schmidhuber (1997); Keskar et al. (2017), is that the flatness of minima of the loss function found by stochastic gradient based methods results in good generalization. This paper argues that most notions of flatness are problematic for deep models and can not be directly applied to explain generalization. Specifically, when focusing on deep networks with rectifier units, we can exploit the particular geometry of parameter space induced by the inherent symmetries that these architectures exhibit to build equivalent models corresponding to arbitrarily sharper minima. Furthermore, if we allow to reparametrize a function, the geometry of its parameters can change drastically without affecting its generalization properties.


To investigate the linguistic patterns captured by MPOBERT, BERT, and ALBERT, we conduct a suite of probing tasks, following the methodology of #REFR .

The results demonstrate that a rank of 8 is sufficient for MPOBERT, which further shows the necessity of layer-specific adapters.

However, we also observe a decrease in the performance of the variant with adapter rank 64.

This illustrates that further increasing the rank may increase the risk of over-fitting in fine-tuning process.

Therefore, we set a rank of 8 for MPOBERT in the main results.

These tasks are designed to evaluate the encoding of surface, syntactic, and semantic information in the models' representations. The results, shown in Fig.

4 , reveal that BERT encodes more local syntax in lower layers and more complex semantics in higher layers, while ALBERT does not exhibit such a clear trend.

However, MPOBERT exhibits similar layer-wise behavior to BERT in some tasks (i.e., task 0,2,4), and improved results in lower layers for others (i.e., task 3) which is similar to ALBERT.

The result demonstrates that MPOBERT captures linguistic information differently than other models, and its layer-wise parameters play an important role in this difference.

Scaling Pre-Trained Language Models To Deeper Via Parameter-Efficient Architecture

Abstract: In this paper, we propose a highly parameterefficient approach to scaling pre-trained language models (PLMs) to a deeper model depth. Unlike prior work that shares all parameters or uses extra blocks, we design a more capable parameter-sharing architecture based on matrix product operator (MPO). MPO decomposition can reorganize and factorize the information of a parameter matrix into two parts: the major part that contains the major information (central tensor) and the supplementary part that only has a small proportion of parameters (auxiliary tensors). Based on such a decomposition, our architecture shares the central tensor across all layers for reducing the model size and meanwhile keeps layerspecific auxiliary tensors (also using adapters) for enhancing the adaptation flexibility. To improve the model training, we further propose a stable initialization algorithm tailored for the MPO-based architecture. Extensive experiments have demonstrated the effectiveness of our proposed model in reducing the model size and achieving highly competitive performance.

What Do You Learn From Context? Probing For Sentence Structure In Contextualized Word Representations

Abstract: Contextualized representation models such as ELMo (Peters et al., 2018a) and BERT (Devlin et al., 2018) have recently achieved state-of-the-art results on a diverse array of downstream NLP tasks. Building on recent token-level probing work, we introduce a novel edge probing task design and construct a broad suite of sub-sentence tasks derived from the traditional structured NLP pipeline. We probe word-level contextual representations from four recent models and investigate how they encode sentence structure across a range of syntactic, semantic, local, and long-range phenomena. We find that existing models trained on language modeling and translation produce strong representations for syntactic phenomena, but only offer comparably small improvements on semantic tasks over a non-contextual baseline.


They found that the PGD attack #REFR performed the best among the compared attacks (We also have the same conclusion in Table 3 ).

By default, we use PSPNet, HRNet, and PSANet to evaluate Pascal VOC, Cityscapes, and ADE20K, respectively.

Table 2 shows the clean pixel accuracy and MIoU (See the end of Section 5.1) of the three models on the three datasets. Compared baselines. We implement our attacks in Py-Torch.

All models are run on a Linux server with 96 core 3.0GHz CPU, 768GB RAM, and 8 Nvidia A100 GPUs.

The source code of our attacks is publicly available at 8 . #OTHEREFR . https://github.com/hszhao/semseg 7. https://github.com/HRNet/HRNet-Semantic-Segmentation 8. https://github.com/randomizedheap/CR Attack • White-box attack algorithms.

#OTHEREFR performed a systematic study to understand the robustness of modern segmentation models against adversarial perturbations.

Thus, in this paper, we mainly use PGD as the base attack and compare it with our CR-PGD attack.

We note that our certified radius can be incorporated into all the existing white-box attacks and we show additional results in Section 6.

Details of the existing attack methods are shown in Appendix A.3.

We mainly evaluate our projected bandit gradient descent (PBGD) attack and certified radius-guided PBGD (CR-PBGD) attack. Parameter settings.

A Certified Radius-Guided Attack Framework To Image Segmentation Models

Abstract: Image segmentation is an important problem in many safety-critical applications such as medical imaging and autonomous driving. Recent studies show that modern image segmentation models are vulnerable to adversarial perturbations, while existing attack methods mainly follow the idea of attacking image classification models. We argue that image segmentation and classification have inherent differences, and design an attack framework specially for image segmentation models. Our goal is to thoroughly explore the vulnerabilities of modern segmentation models, i.e., aiming to misclassify as many pixels as possible under a perturbation budget in both white-box and black-box settings. Our attack framework is inspired by certified radius, which was originally used by defenders to defend against adversarial perturbations to classification models. We are the first, from the attacker perspective, to leverage the properties of certified radius and propose a certified radius guided attack framework against image segmentation models. Specifically, we first adapt randomized smoothing, the state-of-theart certification method for classification models, to derive the pixel's certified radius. A larger certified radius of a pixel means the pixel is theoretically more robust to adversarial perturbations. This observation inspires us to focus more on disrupting pixels with relatively smaller certified radii. Accordingly, we design a pixel-wise certified radius guided loss, when plugged into any existing white-box attack, yields our certified radius-guided white-box attack. Next, we propose the first black-box attack to image segmentation models via bandit. A key challenge is no gradient information is available. To address it, we design a novel gradient estimator, based on bandit feedback, which is queryefficient and provably unbiased and stable. We use this gradient estimator to design a projected bandit gradient descent (PBGD) attack. We further use pixels' certified radii and design a certified radius-guided PBGD (CR-PBGD) attack. We prove our PBGD and CR-PBGD attacks can achieve asymptotically optimal attack performance with an optimal rate. We evaluate our certified-radius guided white-box and black-box attacks on multiple modern image segmentation models and datasets. Our results validate the effectiveness of our certified radius-guided attack framework.

Towards Deep Learning Models Resistant To Adversarial Attacks

Abstract: Recent work has demonstrated that neural networks are vulnerable to adversarial examples, i.e., inputs that are almost indistinguishable from natural data and yet classified incorrectly by the network. In fact, some of the latest findings suggest that the existence of adversarial attacks may be an inherent weakness of deep learning models. To address this problem, we study the adversarial robustness of neural networks through the lens of robust optimization. This approach provides us with a broad and unifying view on much of the prior work on this topic. Its principled nature also enables us to identify methods for both training and attacking neural networks that are reliable and, in a certain sense, universal. In particular, they specify a concrete security guarantee that would protect against any adversary. These methods let us train networks with significantly improved resistance to a wide range of adversarial attacks. They also suggest the notion of security against a first-order adversary as a natural and broad security guarantee. We believe that robustness against such well-defined classes of adversaries is an important stepping stone towards fully resistant deep learning models.


This is not applicable as is to state-of-the-art one-stage object detectors such as YOLO #REFR or DETR (Carion et al., 2020) .

for the softmax), following the principles introduced by #OTHEREFR .

They control the coordinates of the boxes but also the proposal and objectness scores, resulting in more and larger boxes.

Their method relies on the structure of Fast R- CNN (Ren et al., 2017) , the underlying OD model: this has three detection steps with three predictors associated with the proposal, presence and location of a bounding box.

Each component is controlled individually and then combined to attain the desired guarantee.

Their method is an application of the PAC-based calibration of #OTHEREFR .

This is one of the reasons why we opt to model our uncertainty quantification problem directly via CP.

Also, CP requires exchangeable data while concentration-based methods such as #OTHEREFR

(2020) and the more general methods of and Angelopoulos et al.

(2021)require the stronger assumption of data being independently and identically distributed (i.i.d).

Confident Object Detection Via Conformal Prediction And Conformal Risk Control: An Application To Railway Signaling

Abstract: Deploying deep learning models in real-world certified systems requires the ability to provide confidence estimates that accurately reflect their uncertainty. In this paper, we demonstrate the use of the conformal prediction framework to construct reliable and trustworthy predictors for detecting railway signals. Our approach is based on a novel dataset that includes images taken from the perspective of a train operator and state-of-the-art object detectors. We test several conformal approaches and introduce a new method based on conformal risk control. Our findings demonstrate the potential of the conformal prediction framework to evaluate model performance and provide practical guidance for achieving formally guaranteed uncertainty bounds.

You Only Look Once: Unified, Real-Time Object Detection

Abstract: We present YOLO, a new approach to object detection. Prior work on object detection repurposes classifiers to perform detection. Instead, we frame object detection as a regression problem to spatially separated bounding boxes and associated class probabilities. A single neural network predicts bounding boxes and class probabilities directly from full images in one evaluation. Since the whole detection pipeline is a single network, it can be optimized end-to-end directly on detection performance. ::: Our unified architecture is extremely fast. Our base YOLO model processes images in real-time at 45 frames per second. A smaller version of the network, Fast YOLO, processes an astounding 155 frames per second while still achieving double the mAP of other real-time detectors. Compared to state-of-the-art detection systems, YOLO makes more localization errors but is far less likely to predict false detections where nothing exists. Finally, YOLO learns very general representations of objects. It outperforms all other detection methods, including DPM and R-CNN, by a wide margin when generalizing from natural images to artwork on both the Picasso Dataset and the People-Art Dataset.


This result is similar to the one proved in #REFR for team-based modal logic ML( ). Proof.

A formula ϕ is in -disjunctive normal form if it is of the form i∈I α i , where α i are LTL-formulae.

We will next show that every formula of TeamLTL l ( ) can be transformed into equivalent -disjunctive normal form.

The proof proceeds by induction on the structure of formulae.

Note that atomic formulae are already in the normal form and that the case for is trivial. The remaining cases are defined as follows:

(ψ ∧ θ) * := i∈I,j∈J (α ψ i ∧ α θ j ) (ψ ∨ θ) * := i∈I,j∈J (α ψ i ∨ α θ j ) ( ψ) * := i∈I α ψ i (G ψ) * := i∈I G α ψ i (ψ U θ) * := i∈I,j∈J (α ψ i U α θ j ).

where α ψ i and α θ j are the flat formulae in the disjunctive normal forms of ψ and θ respectively, and I and J are the respective index sets.

Suppose ϕ = ψ ∧ θ and that ψ ≡ i∈I α ψ i and θ ≡ i∈J α θ j (induction hypothesis).

Set Semantics For Asynchronous Teamltl: Expressivity And Complexity

Abstract: We introduce and develop a set-based semantics for asynchronous TeamLTL. We consider two canonical logics in this setting: the extensions of TeamLTL by the Boolean disjunction and by the Boolean negation. We establish fascinating connections between the original semantics based on multisets and the new set-based semantics as well as show one of the first positive complexity theoretic results in the temporal team semantics setting. In particular we show that both logics enjoy normal forms that can be utilised to obtain results related to expressivity and complexity (decidability) of the new logics. We also relate and apply our results to recently defined logics whose asynchronicity is formalized via time evaluation functions.

Complexity Of Validity For Propositional Dependence Logics

Abstract: We study the validity problem for propositional dependence logic, modal dependence logic and extended modal dependence logic. We show that the validity problem for propositional dependence logic is NEXPTIME-complete. In addition, we establish that the corresponding problem for modal dependence logic and extended modal dependence logic is NEXPTIME-hard and in NEXPTIME^NP.


Reinforced Transfer network #REFR eliminates outlier samples in source classes through a reinforced data selector by considering both high-level and pixel-level information.

To select the instance via class-level weighting, Selective Adversarial Network #OTHEREFR introduces multiple source discriminators to weigh the source samples according to the class probabilities predicted by the discriminators.

To generate the weighting strategy by instance-level, Importance Weighted Adversarial Nets #OTHEREFR derives the probability of a source example belonging to the target domain and weighs the source examples based on the probability for the domainadversarial network.

Example Transfer Network #OTHEREFR weighs the source samples according to the similarities measured by a domain discriminator and down-weights irrelevant source samples when updating the parameters of the source classifier.

As the key technique toward partial domain adaptation relies on iterative instance selection, recent advancements formulate the instance selection problem into a Markov decision process and adopt deep reinforcement learning algorithms to go through individual instances for partial domain adaptation.

Domain Adversarial Reinforcement Learning #OTHEREFR iteratively selects instances from a candidate set to a finalized set for performing domain adaptation with a tailored reward function based on the domain adversarial learning framework.

However, instance-wise selective transfer methods are not applicable to time series data since there exist temporal dependencies between instances and arbitrary selection of partial instances may fail to consider the dependencies during domain adaptation.

As a result, we propose a contextual domain adaptation framework that focuses on context sampling for individual instances in time series data.

Context-Aware Domain Adaptation For Time Series Anomaly Detection

Abstract: Time series anomaly detection is a challenging task with a wide range of real-world applications. Due to label sparsity, training a deep anomaly detector often relies on unsupervised approaches. Recent efforts have been devoted to time series domain adaptation to leverage knowledge from similar domains. However, existing solutions may suffer from negative knowledge transfer on anomalies due to their diversity and sparsity. Motivated by the empirical study of context alignment between two domains, we aim to transfer knowledge between two domains via adaptively sampling context information for two domains. This is challenging because it requires simultaneously modeling the complex in-domain temporal dependencies and cross-domain correlations while exploiting label information from the source domain. To this end, we propose a framework that combines context sampling and anomaly detection into a joint learning procedure. We formulate context sampling into the Markov decision process and exploit deep reinforcement learning to optimize the time series domain adaptation process via context sampling and design a tailored reward function to generate domain-invariant features that better align two domains for anomaly detection. Experiments on three public datasets show promise for knowledge transfer between two similar domains and two entirely different domains.

Selective Transfer With Reinforced Transfer Network For Partial Domain Adaptation

Abstract: Partial domain adaptation (PDA) extends standard domain adaptation to a more realistic scenario where the target domain only has a subset of classes from the source domain. The key challenge of PDA is how to select the relevant samples in the shared classes for knowledge transfer. Previous PDA methods tackle this problem by re-weighting the source samples based on the prediction of classifier or discriminator, thus discarding the pixel-level information. In this paper, to utilize both high-level and pixel-level information, we propose a reinforced transfer network (RTNet), which is the first work to apply reinforcement learning to address the PDA problem. The RTNet simultaneously mitigates the negative transfer by adopting a reinforced data selector to filter out outlier source classes, and promotes the positive transfer by employing a domain adaptation model to minimize the distribution discrepancy in the shared label space. Extensive experiments indicate that RTNet can achieve state-of-the-art performance for partial domain adaptation tasks on several benchmark datasets. Codes and datasets will be available online.

reinforced data selector
Reinforced Transfer network


For example, graph learning methods such as GNN #REFR take networks as input, and our method can help to handle the semi-supervised prediction setting when the training data are only partially observed.

One limitation of our theory is the strict low-rank assumption; theoretical properties for approximately low-rank models are thus far unknown.

A more general theory in these scenarios would largely expand the method's scope.

As another example, if a sequence of evolving networks is observed (subject to egocentric missingness), a critical but open question concerns how to fit a dynamic network model compatible with this missingness.

We will leave this and other investigations for future work.

There are several applications in which we can potentially embed the current method.

Another application is to use our theory to study the privacy-preserving algorithms for network data #OTHEREFR .

The rank of the model satisifes rank(P 11 ) = rank(P) = K.

Assumption A4 (Well-conditioned model). There exists a constant ψ > 0 such that

1 ψ • np * ≤ σ K (P 11 ) ≤ σ 1 (P 11 ) ≤ ψ • np * 1 ψ • N p * ≤ σ K (P) ≤ σ 1 (P) ≤ ψ • N p *

Fitting Low-Rank Models On Egocentrically Sampled Partial Networks

Abstract: The statistical modeling of random networks has been widely used to uncover interaction mechanisms in complex systems and to predict unobserved links in real-world networks. In many applications, network connections are collected via egocentric sampling: a subset of nodes is sampled first, after which all links involving this subset are recorded; all other information is missing. Compared with the assumption of "uniformly missing at random", egocentrically sampled partial networks require specially designed modeling strategies. Current statistical methods are either computationally infeasible or based on intuitive designs without theoretical justification. Here, we propose an approach to fit general low-rank models for egocentrically sampled networks, which include several popular network models. This method is based on graph spectral properties and is computationally efficient for large-scale networks. It results in consistent recovery of missing subnetworks due to egocentric sampling for sparse networks. To our knowledge, this method offers the first theoretical guarantee for egocentric partial network estimation in the scope of low-rank models. We evaluate the technique on several synthetic and real-world networks and show that it delivers competitive performance in link prediction tasks. 1 INTRODUCTION Massive network data that capture complicated dynamics and interactions in human society, the economy, ecosystems, and biology are now available (Goldenberg et al., 2010; Newman, 2018). The past 15 years have witnessed substantial progress in random network models within the statistics field. Associated efforts have provided countless model options to analyze network data with well-established theories (

Graph Neural Networks: A Review Of Methods And Applications

Abstract: Lots of learning tasks require dealing with graph data which contains rich relation information among elements. Modeling physics system, learning molecular fingerprints, predicting protein interface, and classifying diseases require a model to learn from graph inputs. In other domains such as learning from non-structural data like texts and images, reasoning on extracted structures, like the dependency tree of sentences and the scene graph of images, is an important research topic which also needs graph reasoning models. Graph neural networks (GNNs) are connectionist models that capture the dependence of graphs via message passing between the nodes of graphs. Unlike standard neural networks, graph neural networks retain a state that can represent information from its neighborhood with arbitrary depth. Although the primitive GNNs have been found difficult to train for a fixed point, recent advances in network architectures, optimization techniques, and parallel computation have enabled successful learning with them. In recent years, systems based on variants of graph neural networks such as graph convolutional network (GCN), graph attention network (GAT), gated graph neural network (GGNN) have demonstrated ground-breaking performance on many tasks mentioned above. In this survey, we provide a detailed review over existing graph neural network models, systematically categorize the applications, and propose four open problems for future research.


Such x 1 i are called called a transition equilibrium vector (TEV) (Note in Figure 1 , x #REFR 1 , x 1 2 are TEVs, but x 1 3 is not).

Morse theory gives the answer to the question of when the topology of #|X a | changes as a varies: (Here #|A| denotes the number of connected components of a set A.) See #OTHEREFR , #OTHEREFR , #OTHEREFR for more details.)

#|X a+ε | = #|X a−ε | + 1 for a sufficiently small ε > 0, if and only if, a ∈ {f (

where H q (•) is the q-th homology space and ∼ = implies homotopy equivalent.

• #|X a | remains constant, i.e., #|X a+ε | = #|X a−ε | for a sufficiently small ε > 0, if and only if, a passes the value f (x k ) of an index-k equilibrium vector x k with k > 1.

Improving The Utility Of Differentially Private Clustering Through Dynamical Processing

Abstract: This study aims to alleviate the trade-off between utility and privacy in the task of differentially private clustering. Existing works focus on simple clustering methods, which show poor clustering performance for non-convex clusters. By utilizing Morse theory, we hierarchically connect the Gaussian sub-clusters to fit complex cluster distributions. Because differentially private sub-clusters are obtained through the existing methods, the proposed method causes little or no additional privacy loss. We provide a theoretical background that implies that the proposed method is inductive and can achieve any desired number of clusters. Experiments on various datasets show that our framework achieves better clustering performance at the same privacy level, compared to the existing methods.

Engineering Methods For Differentially Private Histograms: Efficiency Beyond Utility

Abstract: Publishing histograms with $\epsilon$-differential privacy has been studied extensively in the literature. Existing schemes aim at maximizing the utility of the published data, while previous experimental evaluations analyze the privacy/utility trade-off. In this paper we provide the first experimental evaluation of differentially private methods that goes beyond utility, emphasizing also on another important aspect, namely efficiency. Towards this end, we first observe that all existing schemes are comprised of a small set of common blocks. We then optimize and choose the best implementation for each block, determine the combinations of blocks that capture the entire literature, and propose novel block combinations. We qualitatively assess the quality of the schemes based on the skyline of efficiency and utility, i.e., based on whether a method is dominated on both aspects or not. Using exhaustive experiments on four real datasets with different characteristics, we conclude that there are always trade-offs in terms of utility and efficiency. We demonstrate that the schemes derived from our novel block combinations provide the best trade-offs for time critical applications. Our work can serve as a guide to help practitioners engineer a differentially private histogram scheme depending on their application requirements.


It is very similar in spirit to the analysis performed in #REFR , where analogous bounds were obtained directly for the change of centrality scores instead of for their sensitivity.

(5.10) |[L exp (A G , E ij )] uv | ≤ 8 e λmax • m(u, v) λ max − λ min e • (λ max − λ min ) 4m(u, v) + 2(λ max − λ min ) m(u,v) .

The result of Corollary 5.6 can be used to obtain a priori estimates for the sensitivity of individual nodes with respect to modifications in an edge (i, j).

In particular, for the sensitivity of subgraph centrality, an estimate is directly obtained by setting v = u in (5.9) or (5.10).

This mathematically confirms the intuition that nodes are more sensitive to the modification of "nearby" edges than to modifications of edges in other parts of the network.

If the extremal eigenvalues λ min , λ max of A G are not known, one can still obtain decay estimates by inserting suitable bounds.

As a simple example, by Geršgorin's disk theorem we have for any adjacency matrix of an undirected graph that spec(A G ) ⊂ [− deg max , deg max ], where deg max denotes the maximum degree of any node in G.

Thus λ max − λ min ≤ 2 deg max , and for graphs where all nodes have similar degrees (which is, e.g., the case for grid-like graphs and many road networks), the decay estimates obtained from using this bound might still carry meaningful information.

In graphs with highly varying degrees, the estimates obtained this way will typically not accurately capture the actual decay behavior.

Sensitivity Of Matrix Function Based Network Communicability Measures: Computational Methods And A Priori Bounds

Abstract: When analyzing complex networks, an important task is the identification of those nodes which play a leading role for the overall communicability of the network. In the context of modifying networks (or making them robust against targeted attacks or outages), it is also relevant to know how sensitive the network's communicability reacts to changes in certain nodes or edges. Recently, the concept of total network sensitivity was introduced in [O.

On The Stability Of Network Indices Defined By Means Of Matrix Functions

Abstract: Identifying important components in a network is one of the major goals of network analysis. Popular and effective measures of importance of a node or a set of nodes are defined in terms of suitable entries of functions of matrices $f(A)$. These kinds of measures are particularly relevant as they are able to capture the global structure of connections involving a node. However, computing the entries of $f(A)$ requires a significant computational effort. In this work we address the problem of estimating the changes in the entries of $f(A)$ with respect to changes in the edge structure. Intuition suggests that, if the topology of connections in the new graph $\tilde G$ is not significantly distorted, relevant components in $G$ maintain their leading role in $\tilde G$. We propose several bounds giving mathematical reasoning to such intuition and showing, in particular, that the magnitude of the variation of the entry $f(A)_{k\ell}$ decays exponentially with the shortest-path distance in $G$ that separates either $k$ or $\ell$ from the set of nodes touched by the edges that are perturbed. Moreover, we propose a simple method that exploits the computation of $f(A)$ to simultaneously compute the all-pairs shortest-path distances of $G$, with essentially no additional cost. As the nodes whose edge connection tends to change more often or tends to be more often affected by noise have marginal role in the graph and are distant from the most central nodes, the proposed bounds are particularly relevant.


In their code, the authors #REFR implemented a two-stage trick that uses a relatively larger tolerance in early iterations and a smaller tolerance in later iterations to terminate the POGS.

In this section, we conduct numerical experiments to compare our IPL method with existing methods for solving the RPR problem #OTHEREFR . The algorithms that we test include the following ones.

(i) PL: The original proximal linear algorithm proposed by #OTHEREFR where the subproblem (5) is solved by POGS #OTHEREFR .

POGS terminates when both the primal residual and the dual residual are small enough.

In our comparison, we use all the default parameters set by the authors in their codes 2 .

The subgradient method with geometrically decaying step sizes was proposed by #OTHEREFR , and they used this algorithm to solve the RPR (4). One typical iteration of this algorithm is

in which λ 0 > 0, q ∈ (0, 1) are hyper-parameters and

A New Inexact Proximal Linear Algorithm With Adaptive Stopping Criteria For Robust Phase Retrieval

Abstract: This paper considers the robust phase retrieval problem, which can be cast as a nonsmooth and nonconvex optimization problem. We propose a new inexact proximal linear algorithm with the subproblem being solved inexactly. Our contributions are two adaptive stopping criteria for the subproblem. The convergence behavior of the proposed methods is analyzed. Through experiments on both synthetic and real datasets, we demonstrate that our methods are much more efficient than existing methods, such as the original proximal linear algorithm and the subgradient method.

Solving (Most) Of A Set Of Quadratic Equalities: Composite Optimization For Robust Phase Retrieval

Abstract: We develop procedures, based on minimization of the composition $f(x) = h(c(x))$ of a convex function $h$ and smooth function $c$, for solving random collections of quadratic equalities, applying our methodology to real-valued phase retrieval problems. We show that the prox-linear algorithm we develop can solve (robust) phase retrieval problems (even with adversarially faulty measurements) with high probability under appropriate random measurement models as soon as the number of measurements $m$ is a constant factor larger than the dimension $n$ of the signal to be recovered. The algorithm requires essentially no tuning---it requires solution of a sequence of convex problems---and it is implementable without any particular assumptions on the measurements taken. We provide substantial experiments investigating our methods, indicating the practical effectiveness of the procedures and showing that they succeed with high probability as soon as $m / n \ge 2$.


These results are presented in Figure  3 and show the gains in inference time with the XNNPack sparse inference accelerator #REFR [17].

In this section we first evaluate and explain the results observed from applying pruning, post-training quantization and weight clustering independently to CsiNet. These results are summarized in Table I .

We then present the results obtained from combining different model compression techniques together. These results are summarized in Table II .

Next, we investigate the effects of applying different levels of model sparsity percentages and quantization levels -both independently and jointly.

These results presented in Figure 2 provide insights on the trade-offs between accuracy, inference time, and model size -and indicate the sparsity and quantization levels that provide an excellent operating point for CsiNet.

Finally, we demonstrate the need to leverage and implement sparse inference acceleration for hardware implementations of pruned neural networks.

Accelerating And Compressing Deep Neural Networks For Massive Mimo Csi Feedback

Abstract: The recent advances in machine learning and deep neural networks have made them attractive candidates for wireless communications functions such as channel estimation, decoding, and downlink channel state information (CSI) compression. However, most of these neural networks are large and inefficient making it a barrier for deployment in practical wireless systems that require low-latency and low memory footprints for individual network functions. To mitigate these limitations, we propose accelerated and compressed efficient neural networks for massive MIMO CSI feedback. Specifically, we have thoroughly investigated the adoption of network pruning, posttraining dynamic range quantization, and weight clustering to optimize CSI feedback compression for massive MIMO systems. Furthermore, we have deployed the proposed model compression techniques on commodity hardware and demonstrated that in order to achieve inference gains, specialized libraries that accelerate computations for sparse neural networks are required. Our findings indicate that there is remarkable value in applying these model compression techniques and the proposed joint pruning and quantization approach reduced model size by 86.5% and inference time by 76.2% with minimal impact to model accuracy. These compression methods are crucial to pave the way for practical adoption and deployments of deep learningbased techniques in commercial wireless systems. Index Terms-Deep learning, CSI feedback, accelerated neural networks, model compression, massive MIMO.

Fast Sparse Convnets

Abstract: Historically, the pursuit of efficient inference has been one of the driving forces behind research into new deep learning architectures and building blocks. Some recent examples include: the squeeze-and-excitation module, depthwise separable convolutions in Xception, and the inverted bottleneck in MobileNet v2. Notably, in all of these cases, the resulting building blocks enabled not only higher efficiency, but also higher accuracy, and found wide adoption in the field. In this work, we further expand the arsenal of efficient building blocks for neural network architectures; but instead of combining standard primitives (such as convolution), we advocate for the replacement of these dense primitives with their sparse counterparts. While the idea of using sparsity to decrease the parameter count is not new, the conventional wisdom is that this reduction in theoretical FLOPs does not translate into real-world efficiency gains. We aim to correct this misconception by introducing a family of efficient sparse kernels for ARM and WebAssembly, which we open-source for the benefit of the community as part of the XNNPACK library. Equipped with our efficient implementation of sparse primitives, we show that sparse versions of MobileNet v1, MobileNet v2 and EfficientNet architectures substantially outperform strong dense baselines on the efficiency-accuracy curve. On Snapdragon 835 our sparse networks outperform their dense equivalents by $1.3-2.4\times$ -- equivalent to approximately one entire generation of MobileNet-family improvement. We hope that our findings will facilitate wider adoption of sparsity as a tool for creating efficient and accurate deep learning architectures.


A transposition is an operation on a permutation (in general on any sequence of numbers) that consists in swapping two consecutive sequences of the permutation. We define this formally now (c.f. #REFR ). Let σ be a permutation.

σ(n)} Now the main idea of the algorithm for Theorem 2 is to translate the rotation problem for skew trees to an operation on binary strings which we prove in Lemma 7 using the above characterization.

En route this proof, we also show that the skew rotation distance between two skew trees of n internal nodes can be at most n 2 distance.

We also analyse the distance of any given tree T to the nearest skew tree and prove matching upper and lower bounds.

Characterizing Rotation using Transpositions : Given two trees T 1 and T 2 , we can interpret them as permutations σ, τ ∈ S n and study the effect of rotation on these permutations.

We prove that rotation must necessarily be a special kind of transposition operation and hence we call it a tree transposition.

Given three integers i, j, k such that 0 < i < j < k ≤ n, the result of transposition operation δ i,j,k applied to the permutation σ is the following permutation (let q(j) = k + i − j).

For any i ≤ t < q(j), δ i,j,k (t) = σ(t + j − i).

For any q(j) ≤ t < k, δ i,j,k (t) = σ(t+j −k).

On Rotation Distance Of Rank Bounded Trees *

Abstract: Computing the rotation distance between two binary trees with n internal nodes efficiently (in poly(n) time) is a long standing open question in the study of height balancing in tree data structures. In this paper, we initiate the study of this problem bounding the rank of the trees given at the input (defined in [EH89] in the context of decision trees). We define the rank-bounded rotation distance between two given binary trees T 1 and T 2 (with n internal nodes) of rank at most r, denoted by d r (T 1 , T 2), as the length of the shortest sequence of rotations that transforms T 1 to T 2 with the restriction that the intermediate trees must be of rank at most r. We show that the rotation distance problem reduces in polynomial time to the rank bounded rotation distance problem. This motivates the study of the problem in the combinatorial and algorithmic frontiers. Observing that trees with rank 1 coincide exactly with skew trees (binary trees where every internal node has at least one leaf as a child), we show the following results in this frontier : • We present an O(n 2) time algorithm for computing d 1 (T 1 , T 2). That is, when the given trees are skew trees (we call this variant as skew rotation distance problem)-where the intermediate trees are restricted to be skew as well. In particular, our techniques imply that for any two skew trees d(T 1 , T 2) ≤ n 2. * A preliminary version of the paper containing a subset of results appeared in the proceedings of the 28th International Computing and Combinatorics Conference (COCOON 2022).

Sorting By Transpositions Is Difficult

Abstract: In comparative genomics, a transposition is an operation that exchanges two consecutive sequences of genes in a genome. The transposition distance between two genomes, that is, the minimum number of transpositions needed to transform a genome into another, is, according to numerous studies, a relevant evolutionary distance. The problem of computing this distance when genomes are represented by permutations is called the Sorting by Transpositions problem, and has been introduced by Bafna and Pevzner in [Proceedings of the Sixth Annual ACM-SIAM Symposium on Discrete Algorithms, 1995, pp. 614--623]. It has naturally been the focus of a number of studies (see, for instance, [G. Fertin, A. Labarre, I. Rusu, E. Tannier, and S. Vialette, Combinatorics of Genome Rearrangements, The MIT Press, Cambridge, MA, 2009]), but the computational complexity of this problem has remained undetermined for 15 years. In this paper, we answer this long-standing open question by proving that the Sorting by Transpositions problem ...


We mainly use the XLM-RoBERTa large #REFR as the pre-trained language model, which is a widely used encoder.

Generated by feeding a sentence into the encoder, the representation is then input to different classifiers.

Three mainstream NER backend classifiers are adopted: Softmax #OTHEREFR Table 1 : The metrics of our gazetteer in detail. The Total Num.

column means the accurate number of entries in the gazetteer for each track.

Numbers with % denote the coverage rates to entities in the training and validation set.

The Average means coverage rate of the gazetteer with a manual one-to-one matching following and the Average-stat is the coverage rate after adding the statistics-based approach.

Ustc-Nelslip At Semeval-2023 Task 2: Statistical Construction And Dual Adaptation Of Gazetteer For Multilingual Complex Ner

Abstract: This paper describes the system developed by the USTC-NELSLIP team for SemEval-2023 Task 2 Multilingual Complex Named Entity Recognition (MultiCoNER II). A method named Statistical Construction and Dual Adaptation of Gazetteer (SCDAG) is proposed for Multilingual Complex NER. The method first utilizes a statistics-based approach to construct a gazetteer. Secondly, the representations of gazetteer networks and language models are adapted by minimizing the KL divergence between them at both the sentence-level and entity-level. Finally, these two networks are then integrated for supervised named entity recognition (NER) training. The proposed method is applied to XLM-R with a gazetteer built from Wikidata, and shows great generalization ability across different tracks. Experimental results and detailed analysis verify the effectiveness of the proposed method. The official results show that our system ranked 1st on one track (Hindi) in this task.

Unsupervised Cross-Lingual Representation Learning At Scale

Abstract: This paper shows that pretraining multilingual language models at scale leads to significant performance gains for a wide range of cross-lingual transfer tasks. We train a Transformer-based masked language model on one hundred languages, using more than two terabytes of filtered CommonCrawl data. Our model, dubbed XLM-R, significantly outperforms multilingual BERT (mBERT) on a variety of cross-lingual benchmarks, including +13.8% average accuracy on XNLI, +12.3% average F1 score on MLQA, and +2.1% average F1 score on NER. XLM-R performs particularly well on low-resource languages, improving 11.8% in XNLI accuracy for Swahili and 9.2% for Urdu over the previous XLM model. We also present a detailed empirical evaluation of the key factors that are required to achieve these gains, including the trade-offs between (1) positive transfer and capacity dilution and (2) the performance of high and low resource languages at scale. Finally, we show, for the first time, the possibility of multilingual modeling without sacrificing per-language performance; XLM-Ris very competitive with strong monolingual models on the GLUE and XNLI benchmarks. We will make XLM-R code, data, and models publicly available.


Gumbel-VQ #REFR learns valid values for all codebook embeddings, while only a small number of embeddings are actually used for quantization as illustrated in codebook utilization.

With the prevalence of multi-modal image synthesis #OTHEREFR and Transformers #OTHEREFR , unifying data modeling regardless of data modalities has attracted increasing interest from the research communities.

Aiming for a generic data representation across different data modalities, discrete representation learning #OTHEREFR 25] plays a significant role in the unified modeling.

In particular, vector quantization models (e.g., VQ-VAE #OTHEREFR and VQ-GAN #OTHEREFR ) emerge as a promising family for learning generic image representations by discretizing images into discrete tokens. With the Figure 1 .

Visualization of codebook (first row) and illustration of codebook utilization (second row) on ADE20K dataset #OTHEREFR .

VQ-GAN #OTHEREFR severely suffers from codebook collapse as most codebook embeddings are invalid values.

As a comparison, the proposed regularized quantization prevents codebook collapse and achieves full codebook utilization.

The codebook visualization method is provided in the supplementary file.

tokenized representation, generative models such as autoregressive model #OTHEREFR and diffusion model #OTHEREFR can be applied to accommodate the dependency of the sequential tokens for image generation, which is referred as tokenized image synthesis under this context.

Vector quantization models can be broadly grouped into deterministic quantization and stochastic quantization according to the selection of discrete tokens.

Specifically, typical deterministic methods like VQ-GAN #OTHEREFR directly select the best-matching token via Argmin or Argmax, while stochastic methods like Gumbel-VQ #OTHEREFR select a token by stochastically sampling from a predicted token distribution.

Regularized Vector Quantization For Tokenized Image Synthesis

Abstract: Quantizing images into discrete representations has been a fundamental problem in unified generative modeling. Predominant approaches learn the discrete representation either in a deterministic manner by selecting the bestmatching token or in a stochastic manner by sampling from a predicted distribution. However, deterministic quantization suffers from severe codebook collapse and misalignment with inference stage while stochastic quantization suffers from low codebook utilization and perturbed reconstruction objective. This paper presents a regularized vector quantization framework that allows to mitigate above issues effectively by applying regularization from two perspectives. The first is a prior distribution regularization which measures the discrepancy between a prior token distribution and the predicted token distribution to avoid codebook collapse and low codebook utilization. The second is a stochastic mask regularization that introduces stochasticity during quantization to strike a good balance between inference stage misalignment and unperturbed reconstruction objective. In addition, we design a probabilistic contrastive loss which serves as a calibrated metric to further mitigate the perturbed reconstruction objective. Extensive experiments show that the proposed quantization framework outperforms prevailing vector quantization methods consistently across different generative models including autoregressive models and diffusion models.

Vq-Wav2Vec: Self-Supervised Learning Of Discrete Speech Representations

Abstract: We propose vq-wav2vec to learn discrete representations of audio segments through a wav2vec-style self-supervised context prediction task. The algorithm uses either a gumbel softmax or online k-means clustering to quantize the dense representations. Discretization enables the direct application of algorithms from the NLP community which require discrete inputs. Experiments show that BERT pre-training achieves a new state of the art on TIMIT phoneme classification and WSJ speech recognition.


This was solved by using the straight-through gradient estimator (STE) approach #REFR which, in the forward pass of a training step, applies rounding operations to computations involved in the QNN (i.e.

There are two main strategies for training QNNs: post-training quantization and quantization-aware training #OTHEREFR (QAT).

The drawback of the post-training quantization is that it typically results in a drop in the accuracy of the network with a magnitude that depends on the specific dataset and network architecture.

In our work, we use the second approach which is implemented in Larq library [11] .

In QAT, the imprecision of the low-bit fixed-point arithmetic is modeled already during the training process, i.e., the network can adapt to a quantized computation during training.

The challenge for QNNs is that they can not be trained directly with stochastic gradient descent (SGD) like classical NNs.

weights, biases, and arithmetic operations) and in the backward pass, the rounding operations are removed such that the error can backpropagate through the network.

BinaryConnect [9] is one of the first works which uses 1-bit quantization of weights during forward and backward propagation, but not during parameter update to maintain accurate gradient calculation during SGD.

As an observation, the models used in conjuction with BinaryConnect use only linear layers which is suficient for MNIST #OTHEREFR dataset, but convolutional layers for CIFAR-10 #OTHEREFR and SVHN #OTHEREFR . Paper #OTHEREFR binarizes the activations as well.

Similarly, for MNIST dataset they use linear layers, while for CIFAR-10, SVHN and ImageNet [10] they use variants of ConvNet, inspired by VGG #OTHEREFR , with the binarization of the activations.

In XNOR-Net #OTHEREFR , both the weights and the inputs to the convolutional and fully connected layers are approximated with binary values which allows an efficient way of implementing convolutional operations. The paper uses ImageNet dataset in experiments.

Architecturing Binarized Neural Networks For Traffic Sign Recognition

Abstract: Traffic signs support road safety and managing the flow of traffic, hence are an integral part of any vision system for autonomous driving. While the use of deep learning is well-known in traffic signs classification due to the high accuracy results obtained using convolutional neural networks (CNNs) (state of the art is 99.46%), little is known about binarized neural networks (BNNs). Compared to CNNs, BNNs reduce the model size and simplify convolution operations and have shown promising results in computationally limited and energy-constrained devices which appear in the context of autonomous driving. This work presents a bottom-up approach for architecturing BNNs by studying characteristics of the constituent layers. These constituent layers (binarized convolutional layers, max pooling, batch normalization, fully connected layers) are studied in various combinations and with different values of kernel size, number of filters and of neurons by using the German Traffic Sign Recognition Benchmark (GTSRB) for training. As a result, we propose BNNs architectures which achieve more than 90% for GTSRB (the maximum is 96.45%) and an average greater than 80% (the maximum is 88.99%) considering also the Belgian and Chinese datasets for testing. The number of parameters of these architectures varies from 100k to less than 2M. The accompanying material of this paper is publicly available at https://github.com/apostovan21/ BinarizedNeuralNetwork.

Quantized Neural Networks: Training Neural Networks With Low Precision Weights And Activations

Abstract: We introduce a method to train Quantized Neural Networks (QNNs) -- neural networks with extremely low precision (e.g., 1-bit) weights and activations, at run-time. At traintime the quantized weights and activations are used for computing the parameter gradients. During the forward pass, QNNs drastically reduce memory size and accesses, and replace most arithmetic operations with bit-wise operations. As a result, power consumption is expected to be drastically reduced. We trained QNNs over the MNIST, CIFAR-10, SVHN and ImageNet datasets. The resulting QNNs achieve prediction accuracy comparable to their 32-bit counterparts. For example, our quantized version of AlexNet with 1-bit weights and 2-bit activations achieves 51% top-1 accuracy. Moreover, we quantize the parameter gradients to 6-bits as well which enables gradients computation using only bit-wise operation. Quantized recurrent neural networks were tested over the Penn Treebank dataset, and achieved comparable accuracy as their 32-bit counterparts using only 4-bits. Last but not least, we programmed a binary matrix multiplication GPU kernel with which it is possible to run our MNIST QNN 7 times faster than with an unoptimized GPU kernel, without suffering any loss in classification accuracy. The QNN code is available online.


In #REFR a characterization of Fitch graphs in terms of eight forbidden subgraphs on three vertices is given.

It is significant, therefore, whether HGT-edges are found along the path from lca(x, y) to y, the path from lca(x, y) to x, or along both paths. Mathematically, this can be captured in the following manner.

Let T be a rooted tree and H ⊆ E(T ) be a set of HGT-edges.

Then the (directed) Fitch graph F(T, H) has vertex set L(T ) and (x, y) ∈ E(F) if the path from lca T (x, y) to y contains an edge e ∈ H.

A graph G is a (directed) Fitch graph if G = F(T, H) for some tree T and edge set H ⊆ E(T ).

Fitch graphs also have a surprisingly simple characterization in terms of their "closed complementary neighborhoods".

For a directed graph consider If G is a Fitch graph, then C (G) := N (G) ∪ {L} ∪ {{x}|x ∈ L}, i.e., the extension of the closed complementary neighborhoods by the singleton sets and L itself forms a hierarchy on L, which corresponds to a uniquely defined rooted tree T G .

This tree T G is the unique least-resolved tree for G and the set H G of HGT-edges in T G is uniquely determined and consists of all edges xy in T G with x, y ∈ V 0 (T G ) #OTHEREFR .

The directed Fitch graph thus is also informative about the structure of the gene tree.

The pair (T G , H G ) also immediately yields the partition of L into HGT-free subsets.

The Theory Of Gene Family Histories

Abstract: Most genes are part of larger families of evolutionary related genes. The history of gene families typically involves duplications and losses of genes as well as horizontal transfers into other organisms. The reconstruction of detailed gene family histories, i.e., the precise dating of evolutionary events relative to phylogenetic tree of the underlying species has remained a challenging topic despite their importance as a basis for detailed investigations into adaptation and functional evolution of individual members of the gene family. The identification of orthologs, moreover, is a particularly important subproblem of the more general setting considered here. In the last few years, an extensive body of mathematical results has appeared that tightly links orthology, a formal notion of best matches among genes, and horizontal gene transfer. The purpose of this chapter is the broadly outline some of the key mathematical insights and to discuss their implication for practical applications. In particular, we focus on tree-free methods, i.e., methods to infer orthology or horizontal gene transfer as well as gene trees, species trees and reconciliations between them without using a priori knowledge of the underlying trees or statistical models for the inference of phylogenetic trees. Instead, the initial step aims to extract binary relations among genes.

Reconstructing Gene Trees From Fitch'S Xenology Relation

Abstract: Two genes are xenologs in the sense of Fitch if they are separated by at least one horizontal gene transfer event. Horizonal gene transfer is asymmetric in the sense that the transferred copy is distinguished from the one that remains within the ancestral lineage. Hence xenology is more precisely thought of as a non-symmetric relation: y is xenologous to x if y has been horizontally transferred at least once since it diverged from the least common ancestor of x and y. We show that xenology relations are characterized by a small set of forbidden induced subgraphs on three vertices. Furthermore, each xenology relation can be derived from a unique least-resolved edge-labeled phylogenetic tree. We provide a linear-time algorithm for the recognition of xenology relations and for the construction of its least-resolved edge-labeled phylogenetic tree. The fact that being a xenology relation is a heritable graph property, finally has far-reaching consequences on approximation problems associated with xenology relations.


For DYNACL (upper part of the figure), we adopt our proposed augmentation annealing strategy and the momentum technique #REFR in the pretraining phase.

/ * Post-processing phase 2, tune the whole model * / 11 Unlock the parameters in g a 12 forall epoch ∈ {1, 2,

• • • , T } do 13 TRADES (h • g a , {x i }, {ŷ i })

Algorithm 1 demonstrates the pseudo-code of our proposed DYNACL and DYNACL++.

Note that DYNACL++ adds a fast and simple post-processing phase to DYNACL and enjoys higher robustness.

Figure 5 demonstrates the pipeline of the proposed DYNACL and DYNACL++.

Projection heads are added on the top of both branches of the ResNet backbone.

For simplicity, this architecture is not explicitly shown in the figure.

For DYNACL++, we add a simple and fast post-processing phase to DYNACL (bottom part of the figure).

Specifically, we replace the projection head with a linear classification head and adopt our PAT algorithm. Note that both two phases require NO true labels.

Rethinking The Effect Of Data Augmentation In Adversarial Contrastive Learning

Abstract: Recent works have shown that self-supervised learning can achieve remarkable robustness when integrated with adversarial training (AT). However, the robustness gap between supervised AT (sup-AT) and self-supervised AT (self-AT) remains significant. Motivated by this observation, we revisit existing self-AT methods and discover an inherent dilemma that affects self-AT robustness: either strong or weak data augmentations are harmful to self-AT, and a medium strength is insufficient to bridge the gap. To resolve this dilemma, we propose a simple remedy named DYNACL (Dynamic Adversarial Contrastive Learning). In particular, we propose an augmentation schedule that gradually anneals from a strong augmentation to a weak one to benefit from both extreme cases. Besides, we adopt a fast post-processing stage for adapting it to downstream tasks. Through extensive experiments, we show that DYNACL can improve state-of-the-art self-AT robustness by 8.84% under Auto-Attack on the CIFAR-10 dataset, and can even outperform vanilla supervised adversarial training for the first time. Our code is available at https://github.com/PKU-ML/DYNACL.

Momentum Contrast For Unsupervised Visual Representation Learning

Abstract: We present Momentum Contrast (MoCo) for unsupervised visual representation learning. From a perspective on contrastive learning as dictionary look-up, we build a dynamic dictionary with a queue and a moving-averaged encoder. This enables building a large and consistent dictionary on-the-fly that facilitates contrastive unsupervised learning. MoCo provides competitive results under the common linear protocol on ImageNet classification. More importantly, the representations learned by MoCo transfer well to downstream tasks. MoCo can outperform its supervised pre-training counterpart in 7 detection/segmentation tasks on PASCAL VOC, COCO, and other datasets, sometimes surpassing it by large margins. This suggests that the gap between unsupervised and supervised representation learning has been largely closed in many vision tasks.


In recent years, although HERCULES #REFR has emerged, a repair tool designed explicitly for multi-hunk fixes scenarios, it addresses only a specific class of multi-hunk fixes, namely ones with substantially similar patches for each hunk.

It can be seen that cross-fertilization of different problem areas is the future trend, and we emphasize the migratory nature of different techniques.

The multidisciplinary intersection of software engineering, artificial intelligence, system security, etc., will be a research trend that will jointly help APR techniques solve various code problems.

Finding 3: In terms of the complexity of fixing defects, single-hunk bugs seem to be of more concern, and most APR techniques only support the fixing of single-hunk bugs.

This phenomenon reflects that the multi-hunk fixes still is a challenge.

Even though the single-hunk repair tool can iteratively fix multi-hunk bugs, it may use the context with the bugs, so this is not an appropriate solution #OTHEREFR .

In addition, while learningbased solutions are now beginning to focus on multi-hunk fixes #OTHEREFR , there are many limitations, which we will discuss in detail in Finding 18.

Overall, the multi-hunk fixes problem is always a challenge, and the complex dependencies between multiple lines of code make the repair even more difficult.

In the long run, multi-hunk bug fixing is still a direction for future researchers.

Finding 4: From the benchmark datasets used by each APR tool, Defects4J is the most widely used benchmark for software bugs, DeepFix dataset is the most commonly used benchmark for programming errors, CVEfixes and Big-Vul are two of the latest vulnerability fix datasets.

But the amount of data in these datasets is still limited. Defects4J 2.0 contains only 835 bugs.

A Survey On Automated Program Repair Techniques

Abstract: With the rapid development and large-scale popularity of program software, modern society increasingly relies on software systems. However, the problems exposed by software have also come to the fore. Software defect has become an important factor troubling developers. In this context, Automated Program Repair (APR) techniques have emerged, aiming to automatically fix software defect problems and reduce manual debugging work. In particular, benefiting from the advances in deep learning, numerous learning-based APR techniques have emerged in recent years, which also bring new opportunities for APR research. To give researchers a quick overview of APR techniques' complete development and future opportunities, we revisit the evolution of APR techniques and discuss in depth the latest advances in APR research. In this paper, the development of APR techniques is introduced in terms of four different patch generation schemes: search-based, constraint-based, template-based, and learning-based. Moreover, we propose a uniform set of criteria to review and compare each APR tool, summarize the advantages and disadvantages of APR techniques, and discuss the current state of APR development. Furthermore, we introduce the research on the related technical areas of APR that have also provided a strong motivation to advance APR development. Finally, we analyze current challenges and future directions, especially highlighting the critical opportunities that large language models bring to APR research. CCS Concepts: • Software and its engineering → Software testing and debugging.

Harnessing Evolution For Multi-Hunk Program Repair

Abstract: Despite significant advances in automatic program repair (APR)techniques over the past decade, practical deployment remains an elusive goal. One of the important challenges in this regard is the general inability of current APR techniques to produce patches that require edits in multiple locations, i.e., multi-hunk patches. In this work, we present a novel APR technique that generalizes single-hunk repair techniques to include an important class of multi-hunk bugs, namely bugs that may require applying a substantially similar patch at a number of locations. We term such sets of repair locations as evolutionary siblings - similar looking code, instantiated in similar contexts, that are expected to undergo similar changes. At the heart of our proposed method is an analysis to accurately identify a set of evolutionary siblings, for a given bug. This analysis leverages three distinct sources of information, namely the test-suite spectrum, a novel code similarity analysis, and the revision history of the project. The discovered siblings are then simultaneously repaired in a similar fashion. We instantiate this technique in a tool called Hercules and demonstrate that it is able to correctly fix 49 bugs in the Defects4J dataset, the highest of any individual APR technique to date. This includes 15 multi-hunk bugs and overall 13 bugs which have not been fixed by any other technique so far.


In recent years, several novel techniques such as PairNorm layer #REFR and DropEdge Rong et al.

In response, this study aims to address this gap by extending the concept of foundation models to the domain of graph analysis.

The primary goal of this work is to investigate the feasibility and effectiveness of incorporating foundation models in the context of non-Euclidean graphs, thereby contributing to the advancement of foundation models in this area. #OTHEREFR

[2017] , which makes the model scalable by sampling neighbors instead of using all of them. Graph attention network (GAT) #OTHEREFR

[2018] introduces the attention mechanism to learn the weights for each neighbor automatically.

Other techniques like adaptive sampling have been proposed for efficient training #OTHEREFR .

[2020] strategy have been proposed to overcome the oversmoothing problem in GNNs Rusch et al.

Despite the encouraging progress in graph neural networks, to the best of our knowledge, there is little research towards learning task-agnostic foundation models for unversal graph analysis.

But indeed, there are some works that are related to this topic of graph foundation models, such as knowledge transfer and graph reprogramming. In particular, the work in #OTHEREFR

[2021b] introduces a novel knowledge transfer tasks that aims to obtain a multi-talented and lightweight student model, by amalgamating knowledge from heterogeneous teacher GNNs.

Segment Anything In Non-Euclidean Domains: Challenges And Opportunities

Abstract: Figure 1: Illustrations of proposed foundation model in the non-Euclidean domain, which is referred to as "Segment Non-Euclidean Anything (SNA)".

Pairnorm: Tackling Oversmoothing In Gnns

Abstract: The performance of graph neural nets (GNNs) is known to gradually decrease with increasing number of layers. This decay is partly attributed to oversmoothing, where repeated graph convolutions eventually make node embeddings indistinguishable. We take a closer look at two different interpretations, aiming to quantify oversmoothing. Our main contribution is PairNorm, a novel normalization layer that is based on a careful analysis of the graph convolution operator, which prevents all node embeddings from becoming too similar. What is more, PairNorm is fast, easy to implement without any change to network architecture nor any additional parameters, and is broadly applicable to any GNN. Experiments on real-world graphs demonstrate that PairNorm makes deeper GCN, GAT, and SGC models more robust against oversmoothing, and significantly boosts performance for a new problem setting that benefits from deeper GNNs.


B. Evaluation Of Sas And Shs Based On Chaos Engineering

Chaos engineering (CE) is the practice of intentionally causing and studying controlled chaos within software systems operating in realistic environments, with the goal of increasing the systems' resilience and ability to handle unforeseen circumstances #REFR .

The evaluation of self-adaptive and self-healing systems is an important aspect of their design and deployment.

In our previous work #OTHEREFR , we defined the evaluation of self-adaptive and self-healing systems as "an approach to determine if a system meets objectives under operation, identify areas in which the system performs as well as desired or predicted, and provide evidence to the value and applicability of the system."

Several approaches to the evaluation of SAS and SHS include model-based evaluation #OTHEREFR , metric-based evaluation #OTHEREFR , model checking #OTHEREFR , and runtime testing and verification #OTHEREFR each with its benefits and limitations.

However, none of these evaluation approaches are viable for evaluation covering the execution of the system under real-life failure scenarios.

Therefore, to address this gap, we introduce a mechanism that builds on chaos engineering principles.

The core tenets of CE can be outlined as four main principles, which include formulating hypotheses based on the steadystate behavior of systems, introducing variations to real-world events, conducting experiments in a production environment, and automating these experiments for continuous execution.

Our approach evaluates the systems through a systematic process that involves exposing the system to faults and testing its ability to recover from such perturbations.

Chess: A Framework For Evaluation Of Self-Adaptive Systems Based On Chaos Engineering

Abstract: There is an increasing need to assess the correct behavior of self-adaptive and self-healing systems due to their adoption in critical and highly dynamic environments. However, there is a lack of systematic evaluation methods for self-adaptive and self-healing systems. We proposed CHESS, a novel approach to address this gap by evaluating self-adaptive and self-healing systems through fault injection based on chaos engineering (CE). The artifact presented in this paper provides an extensive overview of the use of CHESS through two microservice-based case studies: a smart office case study and an existing demo application called Yelb. It comes with a managing system service, a self-monitoring service, as well as five fault injection scenarios covering infrastructure faults and functional faults. Each of these components can be easily extended or replaced to adopt the CHESS approach to a new case study, help explore its promises and limitations, and identify directions for future research.

A Platform For Automating Chaos Experiments

Abstract: The Netflix video streaming system is composed of many interacting services. In such a large system, failures in individual services are not uncommon. This paper describes the Chaos Automation Platform, a system for running failure injection experiments on the production system to verify that failures in non-critical services do not result in system outages.


Continuous Controlling System Uncertainty

An efficient analysis method can be used to propagate the effect of uncertain parameters in software systems and calculate the robustness of the performance indices, thereby enhancing the flexibility in addressing uncertainty ( #REFR ).

As discussed in Sections 6 and 8, uncertainty has been addressed marginally in the papers selected in this study.

However, both in academia and industry, significant attention is paid to the contexts characterized by a high degree of uncertainty.

To reduce uncertainty and obtain feedback on products/software as soon as possible, it is important to test assumptions and hypotheses in short cycles #OTHEREFR .

Continuous monitoring and frequent re-assessment and rearchitecting are necessary to reduce the uncertainty in CSE and DevOps #OTHEREFR .

Architectural Support For Software Performance In Continuous Software Engineering: A Systematic Mapping Study

Abstract: The continuous software engineering paradigm is gaining popularity in modern development practices, where the interleaving of design and runtime activities is induced by the continuous evolution of software systems. In this context, performance assessment is not easy, but recent studies have shown that architectural models evolving with the software can support this goal. In this paper, we present a mapping study aimed at classifying existing scientific contributions that deal with the architectural support for performancetargeted continuous software engineering. We have applied the systematic mapping methodology to an initial set of 215 potentially relevant papers and selected 66 primary studies that we have analyzed to characterize and classify the current state of research. This classification helps to focus on the main aspects that are being considered in this domain and, mostly, on the emerging findings and implications for future research.

An Efficient Method For Uncertainty Propagation In Robust Software Performance Estimation

Abstract: Software engineers often have to estimate the performance of a software system before having full knowledge of the system parameters, such as workload and operational profile. These uncertain parameters inevitably affect the accuracy of quality evaluations, and the ability to judge if the system can continue to fulfil performance requirements if parameter results are different from expected. Previous work has addressed this problem by modelling the potential values of uncertain parameters as probability distribution functions, and estimating the robustness of the system using Monte Carlo-based methods. These approaches require a large number of samples, which results in high computational cost and long waiting times. To address the computational inefficiency of existing approaches, we employ Polynomial Chaos Expansion (PCE) as a rigorous method for uncertainty propagation and further extend its use to robust performance estimation. The aim is to assess if the software system is robust, i.e., it can withstand possible changes in parameter values, and continue to meet performance requirements. PCE is a very efficient technique, and requires significantly less computations to accurately estimate the distribution of performance indices. Through three very different case studies from different phases of software development and heterogeneous application domains, we show that PCE can accurately (>97\%) estimate the robustness of various performance indices, and saves up to 225 hours of performance evaluation time when compared to Monte Carlo Simulation.


In addition, the proposed CSMN and other competitors are implemented by distributed Tensorflow 1.4, where learning rate, mini-batch, and optimizer, are set as 0.001, 1024, Adam #REFR , respectively. In addition, there are 4 layers in the MLP.

We also adapt MMoE for multi-scenario task by assigning each output for corresponding scenario.

-PLE #OTHEREFR : It contains shared components and task-specific components, and adopts a progressive routing mechanism to extract and separate deeper knowledge, enabling the efficiency of representation across multiple tasks.

-STAR #OTHEREFR : It trains a unified model to serve all scenarios simultaneously, containing shared centered parameters and scenario-specific parameters.

-SAR-Net #OTHEREFR : It predicts users' scenario-specific interests from scenario/target item features and adaptively extracts scenario-specific information across multiple scenarios.

Metrics and Implementation Details To comprehensively evaluate the performance of different methods, we adopt two widely used metrics in recommender systems, i.e., Area Under Curve (AUC) #OTHEREFR and Relative Improvement (RI) #OTHEREFR , where, the larger AUC means better ranking performance, and RI provides an intuitive comparison measure by calculating the relative improvement of a target model over the baseline model.

Logistic loss is used as the loss function for all the competitors, as summarized in Table 2 .

Cold-Start Based Multi-Scenario Ranking Model For Click-Through Rate Prediction

Abstract: Online travel platforms (OTPs), e.g., Ctrip.com or Fliggy.com, can effectively provide travel-related products or services to users. In this paper, we focus on the multi-scenario click-through rate (CTR) prediction, i.e., training a unified model to serve all scenarios. Existing multiscenario based CTR methods struggle in the context of OTP setting due to the ignorance of the cold-start users who have very limited data. To fill this gap, we propose a novel method named Cold-Start based Multiscenario Network (CSMN). Specifically, it consists of two basic components including: 1) User Interest Projection Network (UIPN), which firstly purifies users' behaviors by eliminating the scenario-irrelevant information in behaviors with respect to the visiting scenario, followed by obtaining users' scenario-specific interests by summarizing the purified behaviors with respect to the target item via an attention mechanism; and 2) User Representation Memory Network (URMN), which benefits cold-start users from users with rich behaviors through a memory read and write mechanism. CSMN seamlessly integrates both components in an end-to-end learning framework. Extensive experiments on real-world offline dataset and online A/B test demonstrate the superiority of CSMN over state-of-the-art methods.

Adam: A Method For Stochastic Optimization

Abstract: We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. Empirical results demonstrate that Adam works well in practice and compares favorably to other stochastic optimization methods. Finally, we discuss AdaMax, a variant of Adam based on the infinity norm.


The experiment uses the state-of-the-art exact regret minimizer, CFR+ #REFR , for all double oracle algorithms, and the regret minimization algorithms are initialized with uniform random policies following the default setting.

Oshi Zumo is a board game in which players must repeatedly bid to push a token off the other side of the board #OTHEREFR . The full description of games is in Appendix C.

We evaluate the performance with the exploitability in terms of number of infosets visited and wall time measured in seconds.

The number of visited infosets refers to the total number of nodes traversed by the algorithm, including those encountered during best response (BR) computation.

It is equivalent to the number of touched nodes in the experiment of Stochastic Regret Minimization #OTHEREFR .

It is important to note that the difference between the expanded infosets in XDO paper (McAleer et al., 2021) infosets in the present study is that XDO did not include the infosets during BR computation.

The study begins by analyzing the performance of PDO with various periodicity choices and comparing them with XODO.

Furthermore, the algorithm's performance is compared against baselines, including XDO with restricted game solver CFR+, CFR+ itself, and Extensive-form Fictitious Self-play.

All experiments and algorithm implementations are based on OpenSpiel #OTHEREFR .

In Figure 1 , we present a comparison between the XODO and PDO with periodicity values of m = 1, 10, 50, 100, in terms of exploitability plotted against wall time in seconds.Our results show that in Kuhn poker, PDO algorithms outperform XODO, with all PDO algorithms exhibiting similar performance.

Among the PDO algorithms, we found that PDO with periodicity 100 performs slightly better than the other PDO algorithms.

Regret-Minimizing Double Oracle For Extensive-Form Games

Abstract: By incorporating regret minimization, double oracle methods have demonstrated rapid convergence to Nash Equilibrium (NE) in normal-form games and extensive-form games, through algorithms such as online double oracle (ODO) and extensive-form double oracle (XDO), respectively. In this study, we further examine the theoretical convergence rate and sample complexity of such regret minimization-based double oracle methods, utilizing a unified framework called Regret-Minimizing Double Oracle. Based on this framework, we extend ODO to extensive-form games and determine its sample complexity. Moreover, we demonstrate that the sample complexity of XDO can be exponential in the number of information sets |S|, owing to the exponentially decaying stopping threshold of restricted games. To solve this problem, we propose the Periodic Double Oracle (PDO) method, which has the lowest sample complexity among all existing double oracle methods, being only polynomial in |S|. Empirical evaluations on multiple poker and board games show that PDO achieves significantly faster convergence than previous double oracle algorithms and reaches a competitive level with state-of-theart regret minimization methods.

Solving Large Imperfect Information Games Using Cfr+

Abstract: Counterfactual Regret Minimization and variants (e.g. Public Chance Sampling CFR and Pure CFR) have been known as the best approaches for creating approximate Nash equilibrium solutions for imperfect information games such as poker. This paper introduces CFR$^+$, a new algorithm that typically outperforms the previously known algorithms by an order of magnitude or more in terms of computation time while also potentially requiring less memory.


It is designed for image classification tasks, and it has been trained on a large dataset, ImageNet, to recognize a variety of objects and scenes #REFR .

The pre-trained ResNet model is used as a meaningful residuals extractor, instead of features, from the raw data using identity shortcut connections #OTHEREFR .

• GoogLeNet: also known as Inception-v1, is a deep CNN architecture developed by Google for the imageNet large-scale visual recognition challenge (ILSVRC).

It is widely used for image classification tasks and has achieved state-of-the-art results on a variety of benchmarks #OTHEREFR .

GoogLeNet has been employed as a source model, in #OTHEREFR , by retraining the last few layers of the network to adapt to the new task for DTL-based IDS.

• VGG-16: is a 16 CNN layered architecture that was developed by the visual geometry group at the University of Oxford.

The IDS schemes with a dataset that can be converted to images (matrix) could employ VGG-16 as a backbone model.

In #OTHEREFR , VGG-16 has been used as source model to build an efficient DTL-based IDS.

• Le-Net: is a shallow CNN architecture that was developed by Yann LeCun and his colleagues in the late 1990s.

It was one of the first deep-learning models applied to the task of image recognition and was used for handwritten digit recognition #OTHEREFR .

The main reason behind the popularity of this model was its simple and straightforward CNN architecture.

Deep Transfer Learning Applications In Intrusion Detection Systems: A Comprehensive Review

Abstract: Globally, the external Internet is increasingly being connected to the contemporary industrial control system. As a result, there is an immediate need to protect the network from several threats. The key infrastructure of industrial activity may be protected from harm by using an intrusion detection system (IDS), a preventive measure mechanism, to recognize new kinds of dangerous threats and hostile activities. The most recent artificial intelligence (AI) techniques used to create IDS in many kinds of industrial control networks are examined in this study, with a particular emphasis on IDS-based deep transfer learning (DTL). This latter can be seen as a type of information fusion that merge, and/or adapt knowledge from multiple domains to enhance the performance of the target task, particularly when the labeled data in the target domain is scarce. Publications issued after 2015 were taken into account. These selected publications were divided into three categories: DTL-only and IDS-only are involved in the introduction and background, and DTL-based IDS papers are involved in the core papers of this review. Researchers will be able to have a better grasp of the current state of DTL approaches used in IDS in many different types of networks by reading this review paper. Other useful information, such as the datasets used, the sort of DTL employed, the pre-trained network, IDS techniques, the evaluation metrics including accuracy/F-score and false alarm rate (FAR), and the improvement gained, were also covered. The algorithms, and methods used in several studies, or illustrate deeply and clearly the principle in any DTL-based IDS subcategory are presented to the reader.

Very Deep Convolutional Networks For Large-Scale Image Recognition

Abstract: In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.


We also select objects from the ShapeNetCore dataset #REFR to analyze the capability of OpenAD to generalize to unseen object categories and new affordance labels. Generalization to New Affordance Labels.

We present several examples to demonstrate the generality and flexibility of OpenAD.

Primarily, we use objects from the 3D AffordanceNet #OTHEREFR for our visualizations.

We illustrate several examples showing the ability of OpenAD to generalize to unseen affordance classes in Figure 3 .

In the upper row of Figure 3 , we present the detection results of OpenAD for nine seen affordances on appropriate objects.

As trained on these affordances, OpenAD produces good detection results.

Next, for each object, we feed new affordance labels to the models while keeping the same corresponding object, and present the detection results in the two rows below.

The visualization shows that OpenAD successfully detects the associated regions for the queried new affordance labels, even though the labels are not included in the training set.

Open-Vocabulary Affordance Detection In 3D Point Clouds

Abstract: Affordance detection is a challenging problem with a wide variety of robotic applications. Traditional affordance detection methods are limited to a predefined set of affordance labels, hence potentially restricting the adaptability of intelligent robots in complex and dynamic environments. In this paper, we present the Open-Vocabulary Affordance Detection (OpenAD) method, which is capable of detecting an unbounded number of affordances in 3D point clouds. By simultaneously learning the affordance text and the point feature, OpenAD successfully exploits the semantic relationships between affordances. Therefore, our proposed method enables zero-shot detection and can detect previously unseen affordances without a single annotation example. Intensive experimental results show that OpenAD works effectively on a wide range of affordance detection setups and outperforms other baselines by a large margin. Additionally, we demonstrate the practicality of the proposed OpenAD in real-world robotic applications with a fast inference speed (≈ 100 ms).

Shapenet: An Information-Rich 3D Model Repository

Abstract: We present ShapeNet: a richly-annotated, large-scale repository of shapes represented by 3D CAD models of objects. ShapeNet contains 3D models from a multitude of semantic categories and organizes them under the WordNet taxonomy. It is a collection of datasets providing many semantic annotations for each 3D model such as consistent rigid alignments, parts and bilateral symmetry planes, physical sizes, keywords, as well as other planned annotations. Annotations are made available through a public web-based interface to enable data visualization of object attributes, promote data-driven geometric analysis, and provide a large-scale quantitative benchmark for research in computer graphics and vision. At the time of this technical report, ShapeNet has indexed more than 3,000,000 models, 220,000 models out of which are classified into 3,135 categories (WordNet synsets). In this report we describe the ShapeNet effort as a whole, provide details for all currently available datasets, and summarize future plans.

new affordance labels
ShapeNetCore dataset