In this paper, we adopt the attention strategy to model the motion as a Gaussian distribution similar to #REFR .

Scheduled sampling #OTHEREFR related techniques are also adopted #OTHEREFR , #OTHEREFR for the same purpose, while applying such techniques will significantly slow down the training. Henter et al.

#OTHEREFR adopt normalizing flow in LSTM to get powerful and implicit conditional motion distributions, which greatly improves the diversity of synthesized motions.

Introducing extra temporally-related variables called "phase" provides another option to cope with the over-smoothness problem.

The phase variables can be classified into three categories: the contact-based global phase #OTHEREFR for cyclic motions like running or walking, the local phase #OTHEREFR for acyclic motions like basketball motions, and the phase measured in the frequency domain that is automatically fitted in DeepPhase #OTHEREFR by neural networks.

However, even with the DeepPhase method, different dimensions of phase manifolds still have to be fitted for different motions.

The human skeleton is inherently tree-structured, and different body parts may influence human motion differently.

Training in separate body parts and combining them back is similar to matching curves with piece-wise functions, making the fitting easier.

Many methods #OTHEREFR , #OTHEREFR , #OTHEREFR , #OTHEREFR , #OTHEREFR , #OTHEREFR have applied this idea to different motion synthesis tasks, such as motion prediction and style transfer. Wang et al.

#OTHEREFR decompose the human body into seven parts, which are encoded into a latent space through four fully-connected layers.

Then they decode the motions of the seven body parts from the latent space and concatenate them to form motions for the entire human body.

A Two-Part Transformer Network For Controllable Motion Synthesis

Abstract: Although part-based motion synthesis networks have been investigated to reduce the complexity of modeling heterogeneous human motions, their computational cost remains prohibitive in interactive applications. To this end, we propose a novel two-part transformer network that aims to achieve high-quality, controllable motion synthesis results in real-time. Our network separates the skeleton into the upper and lower body parts, reducing the expensive cross-part fusion operations, and models the motions of each part separately through two streams of auto-regressive modules formed by multi-head attention layers. However, such a design might not sufficiently capture the correlations between the parts. We thus intentionally let the two parts share the features of the root joint and design a consistency loss to penalize the difference in the estimated root features and motions by these two auto-regressive modules, significantly improving the quality of synthesized motions. After training on our motion dataset, our network can synthesize a wide range of heterogeneous motions, like cartwheels and twists. Experimental and user study results demonstrate that our network is superior to state-of-the-art human motion synthesis networks in the quality of generated motions.

Recurrent Network Models For Human Dynamics

Abstract: We propose the Encoder-Recurrent-Decoder (ERD) model for recognition and prediction of human body pose in videos and motion capture. The ERD model is a recurrent neural network that incorporates nonlinear encoder and decoder networks before and after recurrent layers. We test instantiations of ERD architectures in the tasks of motion capture (mocap) generation, body pose labeling and body pose forecasting in videos. Our model handles mocap training data across multiple subjects and activity domains, and synthesizes novel motions while avoiding drifting for long periods of time. For human pose labeling, ERD outperforms a per frame body part detector by resolving left-right body part confusions. For video pose forecasting, ERD predicts body joint displacements across a temporal horizon of 400ms and outperforms a first order motion model based on optical flow. ERDs extend previous Long Short Term Memory (LSTM) models in the literature to jointly learn representations and their dynamics. Our experiments show such representation learning is crucial for both labeling and prediction in space-time. We find this is a distinguishing feature between the spatio-temporal visual domain in comparison to 1D text, speech or handwriting, where straightforward hard coded representations have shown excellent results when directly combined with recurrent units [31].


We use the same byte-pair encoding tokenizer #REFR as CLIP, and truncate input text at maximum 77 tokens.

Refer Appendix B for details about our CLIP baseline.

Our implementation is based on PyTorch #OTHEREFR and timm #OTHEREFR libraries.

We use the Vision Transformer #OTHEREFR as image encoder, considering three models of varying capacity -ViT-S #OTHEREFR , ViT-B, and ViT-L. All use a patch size of 16.

The text encoder is same as CLIP -a 12-layer, 512 dimensions wide Transformer #OTHEREFR language model.

We randomly crop 50-100% area of images and resize them to 224 × 224, following #OTHEREFR .

For text augmentation, we randomly prefix the subreddit names to captions as '{subreddit} : {caption}'.

We initialize image/text encoders in the same style as CLIP, except for one change: we use a sine-cosine position embedding in ViT, like #OTHEREFR He et al., 2022) , and keep it frozen while training.

Hyperbolic Image-Text Representations

Abstract: Visual and linguistic concepts naturally organize themselves in a hierarchy, where a textual concept "dog" entails all images that contain dogs. Despite being intuitive, current large-scale vision and language models such as CLIP (Radford et al., 2021) do not explicitly capture such hierarchy. We propose MERU, a contrastive model that yields hyperbolic representations of images and text. Hyperbolic spaces have suitable geometric properties to embed tree-like data, so MERU can better capture the underlying hierarchy in image-text datasets. Our results show that MERU learns a highly interpretable and structured representation space while being competitive with CLIP's performance on standard multi-modal tasks like image classification and image-text retrieval.

Neural Machine Translation Of Rare Words With Subword Units

Abstract: Neural machine translation (NMT) models typically operate with a fixed vocabulary, but translation is an open-vocabulary problem. Previous work addresses the translation of out-of-vocabulary words by backing off to a dictionary. In this paper, we introduce a simpler and more effective approach, making the NMT model capable of open-vocabulary translation by encoding rare and unknown words as sequences of subword units. This is based on the intuition that various word classes are translatable via smaller units than words, for instance names (via character copying or transliteration), compounds (via compositional translation), and cognates and loanwords (via phonological and morphological transformations). We discuss the suitability of different word segmentation techniques, including simple character ngram models and a segmentation based on the byte pair encoding compression algorithm, and empirically show that subword models improve over a back-off dictionary baseline for the WMT 15 translation tasks English!German and English!Russian by up to 1.1 and 1.3 BLEU, respectively.


This means the continuous-time QHD can converge at any speed along the same evolution path. This result generalizes #REFR Theorem 2.2] .

In this section, we show that QHD is closed under time dilation, i.e., time-dilated QHD evolution is also described by the QHD equation, but with different time-dependent parameters.

The quantum evolution Ψ(x, t) for t ∈ [0, T ] forms a curve in the Hilbert space

We introduce a smooth increasing function τ : [0, T ] → R to represent the reparametrization of time. The reparametrized wave function is

which is the same curve in the Hilbert space L 2 (R d ) but with a different speed.

Quantum Hamiltonian Descent *

Abstract: Gradient descent is a fundamental algorithm in both theory and practice for continuous optimization. Identifying its quantum counterpart would be appealing to both theoretical and practical quantum applications. A conventional approach to quantum speedups in optimization relies on the quantum acceleration of intermediate steps of classical algorithms, while keeping the overall algorithmic trajectory and solution quality unchanged. We propose Quantum Hamiltonian Descent (QHD), which is derived from the path integral of dynamical systems referring to the continuous-time limit of classical gradient descent algorithms, as a truly quantum counterpart of classical gradient methods where the contribution from classically-prohibited trajectories can significantly boost QHD's performance for non-convex optimization. Moreover, QHD is described as a Hamiltonian evolution efficiently simulatable on both digital and analog quantum computers. By embedding the dynamics of QHD into the evolution of the socalled Quantum Ising Machine (including D-Wave and others), we empirically observe that the D-Wave-implemented QHD outperforms a selection of state-of-the-art gradient-based classical solvers and the standard quantum adiabatic algorithm, based on the time-to-solution metric, on non-convex constrained quadratic programming instances up to 75 dimensions. Finally, we propose a "three-phase picture" to explain the behavior of QHD, especially its difference from the quantum adiabatic algorithm.

A Variational Perspective On Accelerated Methods In Optimization

Abstract: Accelerated gradient methods play a central role in optimization, achieving optimal rates in many settings. Although many generalizations and extensions of Nesterov's original acceleration method have been proposed, it is not yet clear what is the natural scope of the acceleration concept. In this paper, we study accelerated methods from a continuous-time perspective. We show that there is a Lagrangian functional that we call the Bregman Lagrangian, which generates a large class of accelerated methods in continuous time, including (but not limited to) accelerated gradient descent, its non-Euclidean extension, and accelerated higher-order gradient methods. We show that the continuous-time limit of all of these methods corresponds to traveling the same curve in spacetime at different speeds. From this perspective, Nesterov's technique and many of its generalizations can be viewed as a systematic way to go from the continuous-time curves generated by the Bregman Lagrangian to a family of discrete-time accelerated algorithms.


We then train a popular code intelligence model, Code2Seq #REFR , using the role-augmented dataset.

#OTHEREFR , who classified variables in programs into role categories based on how they are used (e.g., fixed value, temporary variable, stepper, etc.).

They applied the role concepts in programming learning tasks and found that roles facilitated the abilities of students to mentally process program information and apply programming steps in writing programs.

Inspired by this work, we are interested in how data enrichment with variable role information can impact neural models of code.

In particular, we seek to answer the following question: can explicitly injecting the information about variable roles to the input data help neural models of code achieve better performance and robustness with lesser effort spent on training?

To pursue this investigation, we (1) build a static analyzer to identify two kinds of variable roles, and (2) augment the dataset with explicit roles information.

Next, we compare our trained model against a pretrained version of Code2Seq, released by the creators of the model, who had pretrained it with the original (unaugmented) dataset.

In addition, we evaluate the models' robustness by testing them on noise-induced data (data in which variables in a program are randomly transformed).

For both the role augmentation and noise induction steps, we use the semantic-preserving transformation technique #OTHEREFR . Contributions. This paper provides the following contributions:

• We present a novel study to investigate the impact of augmenting variable roles information in a large code dataset on the performance and robustness of a state-of-the-art neural program model.

• We present and implement a technique to automatically detect certain roles of variables in programs, and consequently embed the roles information into variables in the input programs.

A Study Of Variable-Role-Based Feature Enrichment In Neural Models Of Code

Abstract: Although deep neural models substantially reduce the overhead of feature engineering, the features readily available in the inputs might significantly impact training cost and the performance of the models. In this paper, we explore the impact of an unsuperivsed feature enrichment approach based on variable roles on the performance of neural models of code. The notion of variable roles (as introduced in the works of Sajaniemi et al. [1, 2]) has been found to help students' abilities in programming. In this paper, we investigate if this notion would improve the performance of neural models of code. To the best of our knowledge, this is the first work to investigate how Sajaniemi et al.'s concept of variable roles can affect neural models of code. In particular, we enrich a source code dataset by adding the role of individual variables in the dataset programs, and thereby conduct a study on the impact of variable role enrichment in training the Code2Seq model. In addition, we shed light on some challenges and opportunities in feature enrichment for neural code intelligence models. Index Terms-feature enrichment, variable roles, neural models of code

Code2Seq: Generating Sequences From Structured Representations Of Code

Abstract: The ability to generate natural language sequences from source code snippets has a variety of applications such as code summarization, documentation, and retrieval. Sequence-to-sequence (seq2seq) models, adopted from neural machine translation (NMT), have achieved state-of-the-art performance on these tasks by treating source code as a sequence of tokens. We present ${\rm {\scriptsize CODE2SEQ}}$: an alternative approach that leverages the syntactic structure of programming languages to better encode source code. Our model represents a code snippet as the set of compositional paths in its abstract syntax tree (AST) and uses attention to select the relevant paths while decoding. We demonstrate the effectiveness of our approach for two tasks, two programming languages, and four datasets of up to $16$M examples. Our model significantly outperforms previous models that were specifically designed for programming languages, as well as state-of-the-art NMT models.


This bias can be amplified when differential privacy is incorporated in the model as demonstrated in #REFR .

Given a recommendation evaluation metric M, the use group fairness with respect to groups and is defined as

. Empirically, the user group fairness is measured by

Due to different user activity levels, recommender systems would usually underperform against users who have less historical interactions.

Following the classical 80/20 rule, we select the top 20% users based on their engagement activities as the frequent/active group and the rest as the infrequent/inactive group .

Fairness-Aware Differentially Private Collaborative Filtering

Abstract: Recently, there has been an increasing adoption of differential privacy guided algorithms for privacy-preserving machine learning tasks. However, the use of such algorithms comes with trade-offs in terms of algorithmic fairness, which has been widely acknowledged. Specifically, we have empirically observed that the classical collaborative filtering method, trained by differentially private stochastic gradient descent (DP-SGD), results in a disparate impact on user groups with respect to different user engagement levels. This, in turn, causes the original unfair model to become even more biased against inactive users. To address the above issues, we propose DP-Fair, a two-stage framework for collaborative filtering based algorithms. Specifically, it combines differential privacy mechanisms with fairness constraints to protect user privacy while ensuring fair recommendations. The experimental results, based on Amazon datasets, and user history logs collected from Etsy, one of the largest e-commerce platforms, demonstrate that our proposed method exhibits superior performance in terms of both overall accuracy and user group fairness on both shallow and deep recommendation models compared to vanilla DP-SGD.

Differential Privacy Has Disparate Impact On Model Accuracy

Abstract: Differential privacy (DP) is a popular mechanism for training machine learning models with bounded leakage about the presence of specific points in the training data. The cost of differential privacy is a reduction in the model's accuracy. We demonstrate that this cost is not borne equally: accuracy of DP models drops much more for the underrepresented classes and subgroups. ::: For example, a DP gender classification model exhibits much lower accuracy for black faces than for white faces. Critically, this gap is bigger in the DP model than in the non-DP model, i.e., if the original model is unfair, the unfairness becomes worse once DP is applied. We demonstrate this effect for a variety of tasks and models, including sentiment analysis of text and image classification. We then explain why DP training mechanisms such as gradient clipping and noise addition have disproportionate effect on the underrepresented and more complex subgroups, resulting in a disparate reduction of model accuracy.


The formalism introduced by Arıkan #REFR came with a large panel of theoretical and practical challenges.

The discovery of polar coding enabled 5G and other coding tasks.

A fundamental one is about the reliability of the synthetic channels W α , where W is the underlying physical channel and α is a binary string, and how to order the set {W α : α ∈ {0, 1} m }.

Further, we shall write α γ when W α is favored over W γ for all W .

From general channel models (W being a BMS channel) to particular ones (W being a BEC), several rules were proposed.

Rule Set A (RS-A) was initially used by Mori and Tanaka #OTHEREFR to construct polar codes over BMS channels and is generated by 1 0. Later, Schurch #OTHEREFR and Bardet et al.

#OTHEREFR added 10 01 over BMS channels, which we call Rule Set B (RS-B).

Fast Methods For Ranking Synthetic Becs

Abstract: We gather existing methods that are used to compare and rank the BECs synthesized by a polar code constructor, compare them, and propose new methods that compare synthetic BECs faster.

Channel Polarization: A Method For Constructing Capacity-Achieving Codes For Symmetric Binary-Input Memoryless Channels

Abstract: A method is proposed, called channel polarization, to construct code sequences that achieve the symmetric capacity $I(W)$ of any given binary-input discrete memoryless channel (B-DMC) $W$. The symmetric capacity is the highest rate achievable subject to using the input letters of the channel with equal probability. Channel polarization refers to the fact that it is possible to synthesize, out of $N$ independent copies of a given B-DMC $W$, a second set of $N$ binary-input channels $\{W_N^{(i)}:1\le i\le N\}$ such that, as $N$ becomes large, the fraction of indices $i$ for which $I(W_N^{(i)})$ is near 1 approaches $I(W)$ and the fraction for which $I(W_N^{(i)})$ is near 0 approaches $1-I(W)$. The polarized channels $\{W_N^{(i)}\}$ are well-conditioned for channel coding: one need only send data at rate 1 through those with capacity near 1 and at rate 0 through the remaining. Codes constructed on the basis of this idea are called polar codes. The paper proves that, given any B-DMC $W$ with $I(W)>0$ and any target rate $R<I(W)$, there exists a sequence of polar codes $\{{\mathscr C}_n;n\ge 1\}$ such that ${\mathscr C}_n$ has block-length $N=2^n$, rate $\ge R$, and probability of block error under successive cancellation decoding bounded as $P_{e}(N,R) \le \bigoh(N^{-\frac14})$ independently of the code rate. This performance is achievable by encoders and decoders with complexity $O(N\log N)$ for each.


Experiment 5: T-Sne Visualization Of Generated Data

In the case of the CharacterTrajectories dataset, it is visible that for the DPWGAN #REFR the class distribution shows a clear separation between the classes for both the private and public data (Figure 1a ).

In Figure 1 the results of four datasets are shown.

The first column of each subfigure shows the private train data and the public train data.

Ideally, the data overlaps, as this states that the private and generated (public) dataset are indistinguishable.

The CharacterTrajectories dataset shows this behavior for DPWGAN #OTHEREFR whereas, for the other three datasets, this is not the case.

The second and third columns of the subfigure show the class distribution within the private and public train datasets.

Although the data shows a small offset, this did not hinder the performance by a large margin.

Both models show a similar position of the 20 classes within the space.

Furthermore, the results provide evidence that the dataset generated using GSWGAN #OTHEREFR shows a much better overlap compared to the private data.

In addition, the class separation is much better compared to the DPWGAN #OTHEREFR .

The results provide evidence that the generated data for the CharacterTrajectories and FaceDetection dataset using the GSWGAN #OTHEREFR share the same distribution as the original data (Figure 1b and Figure 1f ).

From Private To Public: Benchmarking Gans In The Context Of Private Time Series Classification

Abstract: Deep learning has proven to be successful in various domains and for different tasks. However, when it comes to private data several restrictions are making it difficult to use deep learning approaches in these application fields. Recent approaches try to generate data privately instead of applying a privacy-preserving mechanism directly, on top of the classifier. The solution is to create public data from private data in a manner that preserves the privacy of the data. In this work, two very prominent GAN-based architectures were evaluated in the context of private time series classification. In contrast to previous work, mostly limited to the image domain, the scope of this benchmark was the time series domain. The experiments show that especially GSWGAN performs well across a variety of public datasets outperforming the competitor DPWGAN. An analysis of the generated datasets further validates the superiority of GSWGAN in the context of time series generation.

Differentially Private Generative Adversarial Network

Abstract: Generative Adversarial Network (GAN) and its variants have recently attracted intensive research interests due to their elegant theoretical foundation and excellent empirical performance as generative models. These tools provide a promising direction in the studies where data availability is limited. One common issue in GANs is that the density of the learned generative distribution could concentrate on the training data points, meaning that they can easily remember training samples due to the high model complexity of deep networks. This becomes a major concern when GANs are applied to private or sensitive data such as patient medical records, and the concentration of distribution may divulge critical patient information. To address this issue, in this paper we propose a differentially private GAN (DPGAN) model, in which we achieve differential privacy in GANs by adding carefully designed noise to gradients during the learning procedure. We provide rigorous proof for the privacy guarantee, as well as comprehensive empirical evidence to support our analysis, where we demonstrate that our method can generate high quality data points at a reasonable privacy level.


E Experiment Details E.1 Dataset Details

UTKFace UTKFace dataset #REFR consists of faces images in the wild with annotations of age, gender, and ethnicity. Features are extracted using CLIP embeddings from images.

The dataset has annotations about the target communities -race, religion, gender, LGBTQ. We used the race group as the group variable.

The total number of instances is 17281, where the training set has 15360 rows and the test set has 1921 rows.

CelebA The CelebA dataset #OTHEREFR is the image datasets that has images of faces with various attributes. Features are extracted using CLIP #OTHEREFR embeddings from images.

We tried gender classification task from this dataset-various other tasks can be constructed. We used "Young" as the group variable.

The total number of instances is 202599, where the training set has 162770 rows and the test set has 39829 rows.

This dataset has attributes of age, gender, race and we chose gender as label (Male:0, Female: 1), solving gender classification.

We used the race as the group variable, encoding "Asian" group as 1, other race groups as 0.

The total number of instances is 23705, where the training set has 18964 rows and the test set has 4741 rows.

Mitigating Source Bias For Fairer Weak Supervision

Abstract: Weak supervision overcomes the label bottleneck, enabling efficient development of training sets. Millions of models trained on such datasets have been deployed in the real world and interact with users on a daily basis. However, the techniques that make weak supervision attractive-such as integrating any source of signal to estimate unknown labels-also ensure that the pseudolabels it produces are highly biased. Surprisingly, given everyday use and the potential for increased bias, weak supervision has not been studied from the point of view of fairness. This work begins such a study. Our departure point is the observation that even when a fair model can be built from a dataset with access to ground-truth labels, the corresponding dataset labeled via weak supervision can be arbitrarily unfair. Fortunately, not all is lost: we propose and empirically validate a model for source unfairness in weak supervision, then introduce a simple counterfactual fairness-based technique that can mitigate these biases. Theoretically, we show that it is possible for our approach to simultaneously improve both accuracy and fairness metrics-in contrast to standard fairness approaches that suffer from tradeoffs. Empirically, we show that our technique improves accuracy on weak supervision baselines by as much as 32% while reducing demographic parity gap by 82.5%.

Age Progression/Regression By Conditional Adversarial Autoencoder

Abstract: If I provide you a face image of mine (without telling you the actual age when I took the picture) and a large amount of face images that I crawled (containing labeled faces of different ages but not necessarily paired), can you show me what I would look like when I am 80 or what I was like when I was 5? The answer is probably a No. Most existing face aging works attempt to learn the transformation between age groups and thus would require the paired samples as well as the labeled query image. In this paper, we look at the problem from a generative modeling perspective such that no paired samples is required. In addition, given an unlabeled image, the generative model can directly produce the image with desired age attribute. We propose a conditional adversarial autoencoder (CAAE) that learns a face manifold, traversing on which smooth age progression and regression can be realized simultaneously. In CAAE, the face is first mapped to a latent vector through a convolutional encoder, and then the vector is projected to the face manifold conditional on age through a deconvolutional generator. The latent vector preserves personalized face features (i.e., personality) and the age condition controls progression vs. regression. Two adversarial networks are imposed on the encoder and generator, respectively, forcing to generate more photo-realistic faces. Experimental results demonstrate the appealing performance and flexibility of the proposed framework by comparing with the state-of-the-art and ground truth.


However, most of them adopt a multi-step adversarial attack, i.e., Projected Gradient Descent (PGD) #REFR , to generate adversarial examples for training.

D EEP neural networks (DNNs) #OTHEREFR , #OTHEREFR , [59] have achieved cutting-edge performance in a lot of problems and tasks.

However, they have been shown to be vulnerable to adversarial perturbations that are often imperceptible to human observers #OTHEREFR , #OTHEREFR .

This vulnerability poses a significant security risk for sensitive applications of DNNs.

To address this issue, a great deal of defense methods #OTHEREFR , #OTHEREFR , #OTHEREFR , #OTHEREFR have been developed to improve the adversarial robustness of DNNs against adversarial attacks.

Standard adversarial training #OTHEREFR , #OTHEREFR , #OTHEREFR , #OTHEREFR has been demonstrated as one of the most effective methods to enhance the robustness of DNNs against adversarial examples.

This approach incurs a significant training cost, which limits the practical application of standard adversarial training.

To reduce training costs, fast adversarial training #OTHEREFR , which can be formulated as a mini-max optimization problem, has been proposed to improve adversarial robustness.

During the inner maximization step, a single-step adversarial attack, such as the Fast Gradient Sign Method (FGSM), is used to generate adversarial examples for training.

However, after just a few training epochs, the model suddenly loses its robustness. This phenomenon is described as catastrophic overfitting #OTHEREFR .

To address this issue, several fast adversarial training variants #OTHEREFR , #OTHEREFR , #OTHEREFR have been proposed from different perspectives to enhance the quality of adversarial examples and prevent overfitting. These approaches arXiv:2304.00202v1 [cs. LG] 1 Apr 2023 Fig. 2 .

Improving Fast Adversarial Training With Prior-Guided Knowledge

Abstract: Fast adversarial training (FAT) is an efficient method to improve robustness. However, the original FAT suffers from catastrophic overfitting, which dramatically and suddenly reduces robustness after a few training epochs. Although various FAT variants have been proposed to prevent overfitting, they require high training costs. In this paper, we investigate the relationship between adversarial example quality and catastrophic overfitting by comparing the training processes of standard adversarial training and FAT. We find that catastrophic overfitting occurs when the attack success rate of adversarial examples becomes worse. Based on this observation, we propose a positive prior-guided adversarial initialization to prevent overfitting by improving adversarial example quality without extra training costs. This initialization is generated by using high-quality adversarial perturbations from the historical training process. We provide theoretical analysis for the proposed initialization and propose a prior-guided regularization method that boosts the smoothness of the loss function. Additionally, we design a prior-guided ensemble FAT method that averages the different model weights of historical models using different decay rates. Our proposed method, called FGSM-PGK, assembles the prior-guided knowledge, i.e., the prior-guided initialization and model weights, acquired during the historical training process. Evaluations of four datasets demonstrate the superiority of the proposed method.

Towards Deep Learning Models Resistant To Adversarial Attacks

Abstract: Recent work has demonstrated that neural networks are vulnerable to adversarial examples, i.e., inputs that are almost indistinguishable from natural data and yet classified incorrectly by the network. In fact, some of the latest findings suggest that the existence of adversarial attacks may be an inherent weakness of deep learning models. To address this problem, we study the adversarial robustness of neural networks through the lens of robust optimization. This approach provides us with a broad and unifying view on much of the prior work on this topic. Its principled nature also enables us to identify methods for both training and attacking neural networks that are reliable and, in a certain sense, universal. In particular, they specify a concrete security guarantee that would protect against any adversary. These methods let us train networks with significantly improved resistance to a wide range of adversarial attacks. They also suggest the notion of security against a first-order adversary as a natural and broad security guarantee. We believe that robustness against such well-defined classes of adversaries is an important stepping stone towards fully resistant deep learning models.


Although DNNs have demonstrated to be successful on many tasks, they are vulnerable to adversarial examples #REFR .

Let x ∈ R d denote an image with dimension d, y denote its ground truth label.

Given a pretrained DNN θ, the adversarial attack is implemented by solving the following optimization problem:

where the L(x, y, θ) denotes the loss function that guides the DNN θ to predict the class of image x as ground-truth y, is the maximum allowed magnitude of perturbation, and x * is the generated adversarial example. Some gradient-based optimizers were proposed to solve (1). Szegedy et al.

were the first to use a box-constrained L-BFGS optimizer #OTHEREFR .

Generating Adversarial Examples With Better Transferability Via Masking Unimportant Parameters Of Surrogate Model

Abstract: Deep neural networks (DNNs) have been shown to be vulnerable to adversarial examples. Moreover, the transferability of the adversarial examples has received broad attention in recent years, which means that adversarial examples crafted by a surrogate model can also attack unknown models. This phenomenon gave birth to the transfer-based adversarial attacks, which aim to improve the transferability of the generated adversarial examples. In this paper, we propose to improve the transferability of adversarial examples in the transfer-based attack via masking unimportant parameters (MUP). The key idea in MUP is to refine the pretrained surrogate models to boost the transfer-based attack. Based on this idea, a Taylor expansionbased metric is used to evaluate the parameter importance score and the unimportant parameters are masked during the generation of adversarial examples. This process is simple, yet can be naturally combined with various existing gradient-based optimizers for generating adversarial examples, thus further improving the transferability of the generated adversarial examples. Extensive experiments are conducted to validate the effectiveness of the proposed MUP-based methods.

Intriguing Properties Of Neural Networks

Abstract: Deep neural networks are highly expressive models that have recently achieved state of the art performance on speech and visual recognition tasks. While their expressiveness is the reason they succeed, it also causes them to learn uninterpretable solutions that could have counter-intuitive properties. In this paper we report two such properties. ::: First, we find that there is no distinction between individual high level units and random linear combinations of high level units, according to various methods of unit analysis. It suggests that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks. ::: Second, we find that deep neural networks learn input-output mappings that are fairly discontinuous to a significant extend. We can cause the network to misclassify an image by applying a certain imperceptible perturbation, which is found by maximizing the network's prediction error. In addition, the specific nature of these perturbations is not a random artifact of learning: the same perturbation can cause a different network, that was trained on a different subset of the dataset, to misclassify the same input.


Additionally, the ensemble design also provides a new tool to quantify the segmentation uncertainty via examining the variation of sub-model softmax probability maps #REFR .

The comparison of our deep ensemble model results against mp-MRI+4 PCs model results highlights the benefit of ensemble learning design.

The four sub-models utilized different input designs that incorporated different 4 RFM PCs.

Since these 4 RFM PCs were uncorrelated, the sub-model results were likely to be complementary, and thus the synthesized result could be further improved.

As shown in Figures 4-6 , owing to the adopted Ostu's method, the ensemble result of the ensemble segmentation result captured all key morphological details shown in the sub-model softmax probability maps.

The simple averaging operation would be unable to retain the complementary image information.

We adopted the entropy metric below to represent this data-related segmentation uncertainty:

Where ̂( , ) represents the intensity at position ( , ) of the softmax probability map .

Figure 8 below shows an example uncertainty map (normalized to 0-100) of WT segmentation.

As seen, high levels of uncertainty are observed at the edges of the WT segment, whereas the segmentation core exhibits minimal levels of uncertainty.

A Radiomics-Incorporated Deep Ensemble Learning Model For Multi-Parametric Mri-Based Glioma Segmentation

Abstract: To develop a deep ensemble learning model with a radiomics spatial encoding execution for improved glioma segmentation accuracy using multi-parametric MRI (mp-MRI). Methods: This model was developed using 369 glioma patients with a 4-modality mp-MRI protocol: T1, contrast-enhanced T1 (T1-Ce), T2, and FLAIR. In each modality volume, a 3D sliding kernel was implemented across the brain to capture image heterogeneity: fifty-six radiomic features were extracted within the kernel, resulting in a 4 th order tensor. Each radiomic feature can then be encoded as a 3D image volume, namely a radiomic feature map (RFM). For each patient, all RFMs extracted from all 4 modalities were processed by the Principal Component Analysis (PCA) for dimension reduction, and the first 4 principal components (PCs) were selected. Next, four deep neural networks as sub-models following the U-Net's architecture were trained for the segmenting of a region-of-interest (ROI): each sub-model utilizes the mp-MRI and 1 of the 4 PCs as a 5channel input for 2D execution. Last, the 4 softmax probability results given by the U-net ensemble were superimposed and binarized by Otsu's method as the segmentation result. Three ensemble models were trained to segment enhancing tumor (ET), tumor core (TC), and whole tumor (WT), respectively. Segmentation results given by the proposed ensemble were compared to the mp-MRI-only U-net results. Results: All 3 radiomics-incorporated deep ensemble learning models were successfully implemented: comparing to mp-MRI-only U-net results, the dice coefficients of ET (0.777→0.817), TC (0.742 → 0.757), and WT (0.823 → 0.854) demonstrated improvements. Accuracy, sensitivity, and specificity results demonstrated the same patterns. 3 Conclusion: The adopted radiomics spatial encoding execution enriches the image heterogeneity information that leads to the successful demonstration of the proposed deep ensemble model, which offers a new tool for mp-MRI based medical image segmentation.

Simple And Scalable Predictive Uncertainty Estimation Using Deep Ensembles

Abstract: Deep neural networks (NNs) are powerful black box predictors that have recently achieved impressive performance on a wide spectrum of tasks. Quantifying predictive uncertainty in NNs is a challenging and yet unsolved problem. Bayesian NNs, which learn a distribution over weights, are currently the state-of-the-art for estimating predictive uncertainty; however these require significant modifications to the training procedure and are computationally expensive compared to standard (non-Bayesian) NNs. We propose an alternative to Bayesian NNs that is simple to implement, readily parallelizable, requires very little hyperparameter tuning, and yields high quality predictive uncertainty estimates. Through a series of experiments on classification and regression benchmarks, we demonstrate that our method produces well-calibrated uncertainty estimates which are as good or better than approximate Bayesian NNs. To assess robustness to dataset shift, we evaluate the predictive uncertainty on test examples from known and unknown distributions, and show that our method is able to express higher uncertainty on out-of-distribution examples. We demonstrate the scalability of our method by evaluating predictive uncertainty estimates on ImageNet.

segmentation uncertainty
sub-model softmax probability


Additionally, the FCN score #REFR is computed to further evaluate details of the performance in the Cityscapes dataset.

We follow recent practices #OTHEREFR and report the Kernel Inception Distance (KID) between feature representations of real and fake images.

The feature extraction #OTHEREFR is done by the Inception v3 model.

This measures the accuracy of the FCN-8s semantic classifier #OTHEREFR (pre-trained on real images) after segmenting the generated images and comparing the results against the labels these images were synthesized from.

Look Atme: The Discriminator Mean Entropy Needs Attention

Abstract: Figure 1. ATME is a GAN where, for each iteration at epoch t, the input images for the generator are corrupted with a noisy representation of the discriminator's decision map at epoch t − 1. The adversarial game enforces this noise to be removed, sending the proper signals for equilibration by encouraging the discriminator to converge towards its maximum entropy state. Convergence is often slower in the cases where the noise significantly affects the semantic content of the input (a) and is faster in the opposite cases (b).

Image-To-Image Translation With Conditional Adversarial Networks

Abstract: We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Indeed, since the release of the pix2pix software associated with this paper, a large number of internet users (many of them artists) have posted their own experiments with our system, further demonstrating its wide applicability and ease of adoption without the need for parameter tweaking. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without hand-engineering our loss functions either.


Additionally, there are 2,885 and 716 complicated tables are extracted to constitute the SciTSR-COMP #REFR .

Consequently, it is extended to the partial version, i.e., ICDAR-2013-P.

ICDAR Considering it is also no training set provided, similar division with ICDAR-2013 dataset is also performed to extend it to the partial version, UNLV-P.

TableBank #OTHEREFR collects 145,000 tables for training and 1,000 ones for testing, where the logical coordinates of cell boxes are provided.

PubTabNet #OTHEREFR directly annotates tables as HTML format, which consists of 339,000 training and 114,000 testing samples.

SciTSR #OTHEREFR is a large-scale dataset containing 12,000 tables for training and 3,000 testing samples, which are mostly from PDF.

To evaluate the capacity of TSR methods under more challenging scenes, in work #OTHEREFR it is further augmented with synthesizing distortion algorithms to simulate distractors brought by capture device, which is called SciTSR-COMP-A.

WTW #OTHEREFR is another complex dataset, which contains 10,970 training images and 3,611 testing images collected from wild complex scenes.

Specifically, it is mainly composed of the tabular objects with explicit boundaries, with information of table id, tabular cell coordinates and row/column information annotated.

Grab What You Need: Rethinking Complex Table Structure Recognition With Flexible Components Deliberation

Abstract: Recently, Table Structure Recognition (TSR) task, aiming at identifying table structure into machine readable formats, has received increasing interest in the community. While impressive success, most single table componentbased methods can not perform well on unregularized table cases distracted by not only complicated inner structure but also exterior capture distortion. In this paper, we raise it as Complex TSR problem, where the performance degeneration of existing methods is attributable to their inefficient component usage and redundant post-processing. To mitigate it, we shift our perspective from table component extraction towards the efficient multiple components leverage, which awaits further exploration in the field. Specifically, we propose a seminal method, termed GrabTab, equipped with newly proposed Component Deliberator. Thanks to its progressive deliberation mechanism, our GrabTab can flexibly accommodate to most complex tables with reasonable components selected but without complicated postprocessing involved. Quantitative experimental results on public benchmarks demonstrate that our method significantly outperforms the state-of-the-arts, especially under more challenging scenes.

Complicated Table Structure Recognition

Abstract: The task of table structure recognition aims to recognize the internal structure of a table, which is a key step to make machines understand tables. Currently, there are lots of studies on this task for different file formats such as ASCII text and HTML. It also attracts lots of attention to recognize the table structures in PDF files. However, it is hard for the existing methods to accurately recognize the structure of complicated tables in PDF files. The complicated tables contain spanning cells which occupy at least two columns or rows. To address the issue, we propose a novel graph neural network for recognizing the table structure in PDF files, named GraphTSR. Specifically, it takes table cells as input, and then recognizes the table structures by predicting relations among cells. Moreover, to evaluate the task better, we construct a large-scale table structure recognition dataset from scientific papers, named SciTSR, which contains 15,000 tables from PDF files and their corresponding structure labels. Extensive experiments demonstrate that our proposed model is highly effective for complicated tables and outperforms state-of-the-art baselines over a benchmark dataset and our new constructed dataset.


We compared our model to YoloV3 #REFR and a fully unfrozen YoloV5. The results are illustrated in Fig. 10 .

As previously indicated, we leveraged YoloV5's architecture to adapt its robust performance and precision.

However, we compared our framework to other cutting-edge architecture, and MagicEye outperforms every comparison.

MagicEye, in particular, employs YoloV5 architecture with a frozen backbone.

By freezing weights, the gradient computations of the network are minimized, thus lowering the model's dimensionality.

This increases efficiency by significantly reducing storage, transfer costs, and computation required for both the forward and backward passes.

In addition, a thorough evaluation was performed on the currency detection and facial recognition capabilities of MagicEye.

As part of the evaluation process, we utilized various metrics such as precision, recall, and f1 score, as demonstrated in Table III .

These metrics serve to assess the robustness of the model and provide a clear understanding of its performance.

In conclusion, the MagicEye system represents a significant advancement in the field of visual aids for the visually impaired.

Its real-time obstacle detection, turn-by-turn navigation, and advanced facial recognition and currency detection capabilities make it a comprehensive solution for the challenges faced by the visually impaired community.

Magiceye: An Intelligent Wearable Towards Independent Living Of Visually Impaired

Abstract: Individuals with visual impairments often face a multitude of challenging obstacles in their daily lives. Vision impairment can severely impair a person's ability to work, navigate, and retain independence. This can result in educational limits, a higher risk of accidents, and a plethora of other issues. To address these challenges, we present MagicEye, a state-of-the-art intelligent wearable device designed to assist visually impaired individuals. MagicEye employs a custom-trained CNNbased object detection model, capable of recognizing a wide range of indoor and outdoor objects frequently encountered in daily life. With a total of 35 classes, the neural network employed by MagicEye has been specifically designed to achieve high levels of efficiency and precision in object detection. The device is also equipped with facial recognition and currency identification modules, providing invaluable assistance to the visually impaired. In addition, MagicEye features a GPS sensor for navigation, allowing users to move about with ease, as well as a proximity sensor for detecting nearby objects without physical contact. In summary, MagicEye is an innovative and highly advanced wearable device that has been designed to address the many challenges faced

Yolov3: An Incremental Improvement

Abstract: We present some updates to YOLO! We made a bunch of little design changes to make it better. We also trained this new network that's pretty swell. It's a little bigger than last time but more accurate. It's still fast though, don't worry. At 320x320 YOLOv3 runs in 22 ms at 28.2 mAP, as accurate as SSD but three times faster. When we look at the old .5 IOU mAP detection metric YOLOv3 is quite good. It achieves 57.9 mAP@50 in 51 ms on a Titan X, compared to 57.5 mAP@50 in 198 ms by RetinaNet, similar performance but 3.8x faster. As always, all the code is online at this https URL


Another popular and simple transfer learning strategy is keeping frozen most of the network #REFR , and training to the target task only some of the model parameters.

Indeed, fine-tuning the already learned meaningful representation to the target task allows one to work with dramatically smaller dataset sizes while keeping the same level of accuracy.

Unfortunately, deep neural networks tend to be sensitive even to tiny distribution shifts #OTHEREFR .

Therefore, performing transfer learning starting from the right initialization and with the right protocol is crucial for success.

One of the major risks is to incur in overfitting #OTHEREFR due to the limited size of the target dataset.

Common strategies to avoid it include damping the learning rate in the fine-tuning stage #OTHEREFR , which forces the network to change its weights only by a small amount.

Transferring the layers directly from another network allows, in principle, filtering out irrelevant information present in the input of both the source and the target data sets, effectively reducing the dimensionality of the problem and increasing data efficiency.

But how should one choose the layers that should be kept frozen? Ideally, one should be able to identify, in the source network, a layer in which the representation is general enough to be meaningful also for the target task, but not too general, as otherwise training it to a specialized task might be hindered by the scarcity of data in the target set.

Given the relevance of transfer learning in realworld applications, it is important to define simple protocols that can be applied without too much finetuning or domain knowledge.

In this work, we propose an approach capable of identifying the layer in which #OTHEREFR to CIFAR-10 (a,d) with 128 images per class and full dataset respectively, from ImageNet #OTHEREFR to Breast Cancer #OTHEREFR with 10 4 images per class (b), from CIFAR-10 to GM with 32 images per class(c).

We can see samples of each dataset in the lower part of the panels, while the curves above -the defrosting profiles -represent the test performance of the network.

Optimal Transfer Protocol By Incremental Layer Defrosting

Abstract: Transfer learning is a powerful tool enabling model training with limited amounts of data. This technique is particularly useful in real-world problems where data availability is often a serious limitation. The simplest transfer learning protocol is based on "freezing" the feature-extractor layers of a network pretrained on a data-rich source task, and then adapting only the last layers to a data-poor target task. This workflow is based on the assumption that the feature maps of the pre-trained model are qualitatively similar to the ones that would have been learned with enough data on the target task. In this work, we show that this protocol is often sub-optimal, and the largest performance gain may be achieved when smaller portions of the pre-trained network are kept frozen. In particular, we make use of a controlled framework to identify the optimal transfer depth, which turns out to depend non-trivially on the amount of available training data and on the degree of source-target task correlation. We then characterize transfer optimality by analyzing the internal representations of two networks trained from scratch on the source and the target task through multiple established similarity measures.

How Transferable Are Features In Deep Neural Networks?

Abstract: Many deep neural networks trained on natural images exhibit a curious phenomenon in common: on the first layer they learn features similar to Gabor filters and color blobs. Such first-layer features appear not to be specific to a particular dataset or task, but general in that they are applicable to many datasets and tasks. Features must eventually transition from general to specific by the last layer of the network, but this transition has not been studied extensively. In this paper we experimentally quantify the generality versus specificity of neurons in each layer of a deep convolutional neural network and report a few surprising results. Transferability is negatively affected by two distinct issues: (1) the specialization of higher layer neurons to their original task at the expense of performance on the target task, which was expected, and (2) optimization difficulties related to splitting networks between co-adapted neurons, which was not expected. In an example network trained on ImageNet, we demonstrate that either of these two issues may dominate, depending on whether features are transferred from the bottom, middle, or top of the network. We also document that the transferability of features decreases as the distance between the base task and target task increases, but that transferring features even from distant tasks can be better than using random features. A final surprising result is that initializing a network with transferred features from almost any number of layers can produce a boost to generalization that lingers even after fine-tuning to the target dataset.


Canonical Surface Mapping #REFR approach demonstrated the ability to learn dense pixelto-3D correspondences with limited supervision by ensuring that the projection of a predicted 3D point back onto the image maps close to the original pixel (cycle of geometric consistency).

The problem can be viewed as an example of a more general area of estimation of dense semantic correspondences.

The latter area includes more established tasks, such as computing pixel-to-pixel correspondences (e.g., SIFT).

The advantage of using a dense mesh as one of the mapping domains is that it contains a complete representation of the object (rather than a partial view), includes 3D information, and provides a convenient modality to select and define manipulation areas and strategy.

To estimate canonical mapping, DensePose #OTHEREFR partitions a 3D shape into surface parts (parameterized as 2D charts) and learns to regress to those parts' uv coordinates (pixel-to-surface correspondences).

Training such a model requires millions of annotated correspondences between images and 3D models.

The work was extended in #OTHEREFR to the prediction of articulations of predefined object parts (parametrized as 6D poses of vertex groups).

Continuous Surface Embeddings (CSE) #OTHEREFR method used as part of our approach, addresses computing dense correspondences by learning to predict, for each foreground pixel in an image, an embedding vector of the corresponding vertex in the object's mesh.

CSE achieves state-of-the-art performance while using fewer annotations and conceptually simpler architecture.

Additionally, the method shows the generalizability potential and the ability to relate and share computation for several object categories.

Canonical Mapping As A General-Purpose Object Descriptor For Robotic Manipulation

Abstract: Perception is an essential part of robotic manipulation in a semi-structured environment. Traditional approaches produce a narrow task-specific prediction (e.g., object's 6D pose), that cannot be adapted to other tasks and is ill-suited for deformable objects. In this paper, we propose using canonical mapping as a near-universal and flexible object descriptor. We demonstrate that common object representations can be derived from a single pre-trained canonical mapping model, which in turn can be generated with minimal manual effort using an automated data generation and training pipeline. We perform a multi-stage experiment using two robot arms that demonstrate the robustness of the perception approach and the ways it can inform the manipulation strategy, thus serving as a powerful foundation for general-purpose robotic manipulation.

Canonical Surface Mapping Via Geometric Cycle Consistency

Abstract: We explore the task of Canonical Surface Mapping (CSM). Specifically, given an image, we learn to map pixels on the object to their corresponding locations on an abstract 3D model of the category. But how do we learn such a mapping? A supervised approach would require extensive manual labeling which is not scalable beyond a few hand-picked categories. Our key insight is that the CSM task (pixel to 3D), when combined with 3D projection (3D to pixel), completes a cycle. Hence, we can exploit a geometric cycle consistency loss, thereby allowing us to forgo the dense manual supervision. Our approach allows us to train a CSM model for a diverse set of classes, without sparse or dense keypoint annotation, by leveraging only foreground mask labels for training. We show that our predictions also allow us to infer dense correspondence between two images, and compare the performance of our approach against several methods that predict correspondence by leveraging varying amount of supervision.


During GAN training, the G network intends to learn the distribution P g over the training data X by simultaneously maximizing the correct decision probability D(x) and minimizing the term (1 − D(G(z))) corresponding to the G network #REFR .

GAN training can be viewed as a minimax two-player game in which the G attempts to learn the training data distribution and estimate a mapping function capable of generating new samples that increase the D's probability of making the wrong decision, while D tries to maximize its probability of making the right decision (i.e., differentiating a training sample from a generated one) #OTHEREFR , #OTHEREFR .

The training results in a unique solution when the D's output is uniformly distributed with a probability of one-half everywhere.

When MLP networks are used for both the G and D networks, backpropagation can be used to train the entire system where the NN representing the G recovers the data distribution without access to the training data and D maximizes the probability of making the right binary decision #OTHEREFR , #OTHEREFR .

If the prior probability of G input z is P z (z), then the generator mapping function can be given by G(z; θ g ) where θ g is the MLP parameters of G.

The D function is given by D(x; θ d ) for input x and a single output D(x), which represents the probability that x came from the training data and not the G #OTHEREFR .

The GAN minimax optimization problem can be described by the following objective function,

min G max D V (D, G) = E x∼Pd(x) {log[D(x)]} + E z∼Pz(z) {log[1 − D(G(z))]}, (15)

The optimum point is reached when the G perfectly recovers the training data distribution (i.e.

Improving Rf-Dna Fingerprinting Performance In An Indoor Multipath Environment Using Semi-Supervised Learning

Abstract: The number of Internet of Things (IoT) deployments is expected to reach 75.4 billion by 2025. Roughly 70% of all IoT devices employ weak or no encryption; thus, putting them and their connected infrastructure at risk of attack by devices that are wrongly authenticated or not authenticated at all. A physical layer-based security approach-known as Specific Emitter Identification (SEI)-has been proposed and is being pursued as a viable IoT security mechanism. SEI is advantageous because it is a passive technique that exploits inherent and distinct features that are unintentionally imparted upon the signal during its formation and transmission within and by the IoT device's Radio Frequency (RF) front-end. SEI's passive exploitation of unintentional signal features removes any need to modify the IoT device, which makes it ideal for existing and future IoT deployments. Despite the amount of SEI research conducted there remains challenges that must be addressed to make SEI a viable IoT security approach. One of these challenges is the extraction of SEI features from signals collected under multipath fading conditions. Multipath corrupts the inherent SEI exploited features that are used to discriminate one IoT device from another; thus, degrading authentication performance and increasing the chance of attack. This work presents two semi-supervised Deep Learning (DL) equalization approaches and compares their performance with the current state of the art. The two approaches are the Conditional Generative Adversarial Network (CGAN) and Joint Convolutional Auto-Encoder and Convolutional Neural Network (JCAECNN). Both approaches learn the channel distribution to enable multipath correction while simultaneously preserving the SEI exploited signal features. CGAN and JCAECNN performance is assessed using a Rayleigh fading channel under degrading SNR, up to thirty-two IoT devices, as well as two publicly available signal sets. The JCAECNN improves SEI performance by 10% beyond that of the current state of the art.

Generative Adversarial Nets

Abstract: We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G. The training procedure for G is to maximize the probability of D making a mistake. This framework corresponds to a minimax two-player game. In the space of arbitrary functions G and D, a unique solution exists, with G recovering the training data distribution and D equal to ½ everywhere. In the case where G and D are defined by multilayer perceptrons, the entire system can be trained with backpropagation. There is no need for any Markov chains or unrolled approximate inference networks during either training or generation of samples. Experiments demonstrate the potential of the framework through qualitative and quantitative evaluation of the generated samples.


It was shown in #REFR that the introduced quadric loss in combination with the Chamfer loss improves 3D reconstruction accuracy by preserving sharp edges of scene objects.

The authors utilize the model for lidars and cameras calibration scenario.

However, the application of the model in localization and mapping scenarios was not studied in their paper.

As we pose the task of global point cloud map consistency estimation as optimization problem, a differentiable loss function is required as minimization objective.

The Chamfer loss introduced in #OTHEREFR is one of the most famous differentiable function for point sets.

It is usually utilized to evaluate a distance between two point clouds, thus is typically suitable for supervised tasks.

The loss is designed to reduce quadric error between the reconstructed points and the input surface.

However, as it was mentioned by the authors of #OTHEREFR , the quadric loss can not preserve the input point distribution, since the quadric loss is zero anywhere on the plane.

On the contrary, in order to reduce the lidar measurements bias caused by high incidence angle, we target to correct mainly the points which belong to planar surfaces.

Therefore a designed global map consistency loss should be able to encourage points from different scans that represent such objects as walls, floors, and roads to belong to the same plane.

Moreover, as shown in the next Section III, we do not require ground-truth data to evaluate the point cloud consistency, and the introduced losses are computed directly on the global map independently on the number of lidar scans and view-points forming the input point cloud.

Self-Supervised Depth Correction Of Lidar Measurements From Map Consistency Loss

Abstract: Depth perception is considered an invaluable source of information in the context of 3D mapping and various robotics applications. However, point cloud maps acquired using consumer-level light detection and ranging sensors (lidars) still suffer from bias related to local surface properties such as measuring beam-to-surface incidence angle, distance, texture, reflectance, or illumination conditions. This fact has recently motivated researchers to exploit traditional filters, as well as the deep learning paradigm, in order to suppress the aforementioned depth sensors error while preserving geometric and map consistency details. Despite the effort, depth correction of lidar measurements is still an open challenge mainly due to the lack of clean 3D data that could be used as ground truth. In this paper, we introduce two novel point cloud map consistency losses, which facilitate self-supervised learning on real data of lidar depth correction models. Specifically, the models exploit multiple point cloud measurements of the same scene from different viewpoints in order to learn to reduce the bias based on the constructed map consistency signal. Complementary to the removal of the bias from the measurements, we demonstrate that the depth correction models help to reduce localization drift. Additionally, we release a data set that contains point cloud data captured in an indoor corridor environment with precise localization and ground truth mapping information 1 .

Learning Embedding Of 3D Models With Quadric Loss

Abstract: Sharp features such as edges and corners play an important role in the perception of 3D models. In order to capture them better, we propose quadric loss, a point-surface loss function, which minimizes the quadric error between the reconstructed points and the input surface. Computation of Quadric loss is easy, efficient since the quadric matrices can be computed apriori, and is fully differentiable, making quadric loss suitable for training point and mesh based architectures. Through extensive experiments we show the merits and demerits of quadric loss. When combined with Chamfer loss, quadric loss achieves better reconstruction results as compared to any one of them or other point-surface loss functions.

3D reconstruction accuracy
introduced quadric loss


In UCF-QNRF dataset #REFR , it costs over 2000 human hours to annotate only 1535 images, in which the average cost per image is over one hour.

In addition, #OTHEREFR proposed Mean Count Proxies to select the best count proxy for each interval to represent its count value during inference.

#OTHEREFR introduces transformer into crowd localization, proposing a crowd localization transformer to directly predict the localization.

#OTHEREFR proposes a local matching point-based framework to alleviate instability.

These fully supervised methods achieve incredible performance on various datasets.

However, the complete annotation for a crowd image is time-consuming and laborious.

Crowd Counting With Sparse Annotation

Abstract: This paper presents a new annotation method called Sparse Annotation (SA) for crowd counting, which reduces human labeling efforts by sparsely labeling individuals in an image. We argue that sparse labeling can reduce the redundancy of full annotation and capture more diverse information from distant individuals that is not fully captured by Partial Annotation methods. Besides, we propose a pointbased Progressive Point Matching network (PPM) to better explore the crowd from the whole image with sparse annotation, which includes a Proposal Matching Network (PMN) and a Performance Restoration Network (PRN). The PMN generates pseudo-point samples using a basic point classifier, while the PRN refines the point classifier with the pseudo points to maximize performance. Our experimental results show that PPM outperforms previous semisupervised crowd counting methods with the same amount of annotation by a large margin and achieves competitive performance with state-of-the-art fully-supervised methods.

Composition Loss For Counting, Density Map Estimation And Localization In Dense Crowds

Abstract: With multiple crowd gatherings of millions of people every year in events ranging from pilgrimages to protests, concerts to marathons, and festivals to funerals; visual crowd analysis is emerging as a new frontier in computer vision. In particular, counting in highly dense crowds is a challenging problem with far-reaching applicability in crowd safety and management, as well as gauging political significance of protests and demonstrations. In this paper, we propose a novel approach that simultaneously solves the problems of counting, density map estimation and localization of people in a given dense crowd image. Our formulation is based on an important observation that the three problems are inherently related to each other making the loss function for optimizing a deep CNN decomposable. Since localization requires high-quality images and annotations, we introduce UCF-QNRF dataset that overcomes the shortcomings of previous datasets, and contains 1.25 million humans manually marked with dot annotations. Finally, we present evaluation measures and comparison with recent deep CNN networks, including those developed specifically for crowd counting. Our approach significantly outperforms state-of-the-art on the new dataset, which is the most challenging dataset with the largest number of crowd annotations in the most diverse set of scenes.


The results also confirm the existing observation in literature #REFR that the instance-wise perturbations cannot improve the standard accuracy for the downstream tasks.

It adds pixel-wise perturbations on the raw inputs that maximize the loss to make the worst-case attacks, and uses the resultant perturbed inputs to train the backbone network.

We extend the FGSM #OTHEREFR that applies instance-wise perturbations to image pixels.

It follows the classic FGSM except the MAE reconstruction loss with the multi-crop tokenization in MAE+ are adopted to compute the adversarial perturbations.

Table 7 reports the fine-tuning results of the FGSM with various for ∞ constraint over 400 epochs of pretraining ViT-B. FGSM does not perform better than MAE+ (83.51%).

This suggests that a straight extension of FGSM cannot improve the accuracy of a MIM-pretrained model in downstream tasks.

Adpe: Adversarial Positional Embeddings For Pretraining Vision Transformers Via Mae+

Abstract: Unsupervised learning of vision transformers seeks to pretrain an encoder via pretext tasks without labels. Among them is the Masked Image Modeling (MIM) aligned with pretraining of language transformers by predicting masked patches as a pretext task. A criterion in unsupervised pretraining is the pretext task needs to be sufficiently hard to prevent the transformer encoder from learning trivial low-level features not generalizable well to downstream tasks. For this purpose, we propose an Adversarial Positional Embedding (AdPE) approach-It distorts the local visual structures by perturbing the position encodings so that the learned transformer cannot simply use the locally correlated patches to predict the missing ones. We hypothesize that it forces the transformer encoder to learn more discriminative features in a global context with stronger generalizability to downstream tasks. We will consider both absolute and relative positional encodings, where adversarial positions can be imposed both in the embedding mode and the coordinate mode. We will also present a new MAE+ baseline that brings the performance of the MIM pretraining to a new level with the AdPE. The experiments demonstrate that our approach can improve the fine-tuning accuracy of MAE by 0.8% and 0.4% over 1600 epochs of pretraining ViT-B and ViT-L on Imagenet1K. For the transfer learning task, it outperforms the MAE with the ViT-B backbone by 2.6% in mIoU on ADE20K, and by 3.2% in AP bbox and 1.6% in AP mask on COCO, respectively. These results are obtained with the AdPE being a pure MIM approach that does not use any extra models or external datasets for pretraining.

Robustness May Be At Odds With Accuracy

Abstract: We show that there exists an inherent tension between the goal of adversarial robustness and that of standard generalization. Specifically, training robust models may not only be more resource-consuming, but also lead to a reduction of standard accuracy. We demonstrate that this trade-off between the standard accuracy of a model and its robustness to adversarial perturbations provably exists even in a fairly simple and natural setting. These findings also corroborate a similar phenomenon observed in practice. Further, we argue that this phenomenon is a consequence of robust classifiers learning fundamentally different feature representations than standard classifiers. These differences, in particular, seem to result in unexpected benefits: the representations learned by robust models tend to align better with salient data characteristics and human perception.


In the results on cross-lingual classification in #REFR , performance on Arabic and Chinese is obviously worse than in German and Spanish.

Baseline model XLM-R-Base has the best performance when the questions are set to German and Spanish.

Results on cross-lingual classification reported in #OTHEREFR indicate that the performance on Spanish and German are also the highest except English.

We can also see from Fig.5 that the token coverage of these two languages is only a little different from the others.

Take all these into consideration, and we think the limited improvement is caused by the deep understanding of German and Spanish in the baseline PLM.

Significantly, in the Arabic and Chinese cases, the baseline performance is the worst, indicating less knowledge storage and shallow understanding of the two languages.

In some cases, the cough may not be present or may only occur occasionally and may not be productive.

Some people with COPD attribute the symptoms to a "smoker's cough".

Sputum may be swallowed or spat out, depending often on social and cultural factors.

Vigorous coughing may lead to rib fractures or a brief loss of consciousness.

Those with COPD often have a history of "common colds" that last a long time.

Bridging The Language Gap: Knowledge Injected Multilingual Question Answering

Abstract: Question Answering (QA) is the task of automatically answering questions posed by humans in natural languages. There are different settings to answer a question, such as abstractive, extractive, boolean, and multiple-choice QA. As a popular topic in natural language processing tasks, extractive question answering task (extractive QA) has gained extensive attention in the past few years. With the continuous evolvement of the world, generalized cross-lingual transfer (G-XLT), where question and answer context are in different languages, poses some unique challenges over cross-lingual transfer (XLT), where question and answer context are in the same language. With the boost of corresponding development of related benchmarks, many works have been done to improve the performance of various language QA tasks. However, only a few works are dedicated to the G-XLT task. In this work, we propose a generalized crosslingual transfer framework to enhance the model's ability to understand different languages. Specifically, we first assemble triples from different languages to form multilingual knowledge. Since the lack of knowledge between different languages greatly limits models' reasoning ability, we further design a knowledge injection strategy via leveraging link prediction techniques to enrich the model storage of multilingual knowledge. In this way, we can profoundly exploit rich semantic knowledge. Experiment results on real-world datasets MLQA demonstrate that the proposed method can improve the performance by a large margin, outperforming the baseline method by 13.18%/12.00% F1/EM on average. Our code and models will be open-sourced.

Unsupervised Cross-Lingual Representation Learning At Scale

Abstract: This paper shows that pretraining multilingual language models at scale leads to significant performance gains for a wide range of cross-lingual transfer tasks. We train a Transformer-based masked language model on one hundred languages, using more than two terabytes of filtered CommonCrawl data. Our model, dubbed XLM-R, significantly outperforms multilingual BERT (mBERT) on a variety of cross-lingual benchmarks, including +13.8% average accuracy on XNLI, +12.3% average F1 score on MLQA, and +2.1% average F1 score on NER. XLM-R performs particularly well on low-resource languages, improving 11.8% in XNLI accuracy for Swahili and 9.2% for Urdu over the previous XLM model. We also present a detailed empirical evaluation of the key factors that are required to achieve these gains, including the trade-offs between (1) positive transfer and capacity dilution and (2) the performance of high and low resource languages at scale. Finally, we show, for the first time, the possibility of multilingual modeling without sacrificing per-language performance; XLM-Ris very competitive with strong monolingual models on the GLUE and XNLI benchmarks. We will make XLM-R code, data, and models publicly available.


These methods have mainly focused on classification models, and ensemble attacks on dense prediction tasks such as object detection and semantic segmentation are relatively less studied, especially for targeted attacks #REFR . Attacks against object detectors and segmentation.

Transfer-based attacks rely on the assumption that surrogate models share similarities with the victim model, such that an adversarial example generated for the surrogate model can also fool the victim model.

Query-based methods generate attacks by searching the adversarial examples space based on the feedback obtained from the victim model through queries.

They can often achieve higher success rate but may require a large number of queries. Ensemble-based attacks.

Ensemble-based attacks leverage the idea of transfer attack and assume that if an adversarial example can fool multiple models simultaneously, the chances of fooling an unseen model are higher #OTHEREFR .

Recently, some methods have combined ensemble-based transfer attacks with limited feedback from the victim models to improve the overall success rate #OTHEREFR .

Dense (pixel-level) prediction tasks such as object detection and semantic segmentation have higher task complexities #OTHEREFR compared to classification tasks.

Existing attacks on object detectors mainly focus on whitebox setting, although there are a few exceptions #OTHEREFR .

A recent study #OTHEREFR generates blackbox attacks on object detectors by using a surrogate ensemble and context-aware attack-based queries.

Another approach #OTHEREFR trains a generative model to generate transferable attacks.

While some patch-based attacks #OTHEREFR are effective, the patches are easily noticeable.

Ensemble-Based Blackbox Attacks On Dense Prediction

Abstract: We propose an approach for adversarial attacks on dense prediction models (such as object detectors and segmentation). It is well known that the attacks generated by a single surrogate model do not transfer to arbitrary (blackbox) victim models. Furthermore, targeted attacks are often more challenging than the untargeted attacks. In this paper, we show that a carefully designed ensemble can create effective attacks for a number of victim models. In particular, we show that normalization of the weights for individual models plays a critical role in the success of the attacks. We then demonstrate that by adjusting the weights of the ensemble according to the victim model can further improve the performance of the attacks. We performed a number of experiments for object detectors and segmentation to highlight the significance of the our proposed methods. Our proposed ensemble-based method outperforms existing blackbox attack methods for object detection and segmentation. Finally we show that our proposed method can also generate a single perturbation that can fool multiple blackbox detection and segmentation models simultaneously. Code is available at https://github.com/CSIPlab/EBAD.

Making An Invisibility Cloak: Real World Adversarial Attacks On Object Detectors

Abstract: We present a systematic study of adversarial attacks on state-of-the-art object detection frameworks. Using standard detection datasets, we train patterns that suppress the objectness scores produced by a range of commonly used detectors, and ensembles of detectors. Through extensive experiments, we benchmark the effectiveness of adversarially trained patches under both white-box and black-box settings, and quantify transferability of attacks between datasets, object classes, and detector models. Finally, we present a detailed study of physical world attacks using printed posters and wearable clothes, and rigorously quantify the performance of such attacks with different metrics.


By comparing the results of EIMP in #REFR and 3, we see that the number of preserved keypoints are based on the number of potential inliers in the image pair: more potential inliers result in more retrained keypoints. That is because our sampling strategy is fully adaptive.

Besides, the number of keypoints in each image are shown at the top of each pair.

For simple case (1), all methods give a large number of inliers, but IMP and EIMP report smaller pose errors even when using fewer inliers.

Note that instead of keeping all keypoints, EIMP effectively reduces the number of keypoints from 2,0000 to 865 and 255 in the iteration process, significantly decreasing the time complexity for self and cross attention computation 7.

For tough cases ((2), (3)), due to large viewpoint and illumination changes, SuperGlue* and SGMNet fail to report comparable number of inliers to our models, resulting in higher rotation and translation errors.

In contrast, our models still progressively increase the number of inliers from different regions in the iteration process. These well distributed inliers lead to smaller pose errors.

Imp: Iterative Matching And Pose Estimation With Adaptive Pooling

Abstract: Previous methods solve feature matching and pose estimation using a two-stage process by first finding matches and then estimating the pose. As they ignore the geometric relationships between the two tasks, they focus on either improving the quality of matches or filtering potential outliers, leading to limited efficiency or accuracy. In contrast, we propose an iterative matching and pose estimation framework (IMP) leveraging the geometric connections between the two tasks: a few good matches are enough for a roughly accurate pose estimation; a roughly accurate pose can be used to guide the matching by providing geometric constraints. To this end, we implement a geometry-aware recurrent attention-based module which jointly outputs sparse matches and camera poses. Specifically, for each iteration, we first implicitly embed geometric information into the module via a pose-consistency loss, allowing it to predict geometry-aware matches progressively. Second, we introduce an efficient IMP, called EIMP, to dynamically discard keypoints without potential matches, avoiding redundant updating and significantly reducing the quadratic time complexity of attention computation in transformers. Experiments on YFCC100m, Scannet, and Aachen Day-Night datasets demonstrate that the proposed method outperforms previous approaches in terms of accuracy and efficiency.

Netvlad: Cnn Architecture For Weakly Supervised Place Recognition

Abstract: We tackle the problem of large scale visual place recognition, where the task is to quickly and accurately recognize the location of a given query photograph. We present the following four principal contributions. First, we develop a convolutional neural network (CNN) architecture that is trainable in an end-to-end manner directly for the place recognition task. The main component of this architecture, NetVLAD, is a new generalized VLAD layer, inspired by the “Vector of Locally Aggregated Descriptors” image representation commonly used in image retrieval. The layer is readily pluggable into any CNN architecture and amenable to training via backpropagation. Second, we create a new weakly supervised ranking loss, which enables end-to-end learning of the architecture's parameters from images depicting the same places over time downloaded from Google Street View Time Machine. Third, we develop an efficient training procedure which can be applied on very large-scale weakly labelled tasks. Finally, we show that the proposed architecture and training procedure significantly outperform non-learnt image representations and off-the-shelf CNN descriptors on challenging place recognition and image retrieval benchmarks.


Comparison With Existing Theoretical Predictions

Those of Item (iv) are not obviously consistent with our experimental data, although we refrain from saying that they are inconsistent since the analysis of #REFR involves a series of inequalities, and it is unclear how the predictions would change if one used slightly different approximations.

(ii) #OTHEREFR predict that adversarial success is a function of ( dim V dim X ) 1 2 (for p = 2).

(iii) From the analysis in #OTHEREFR one can predict that adversarial success is a function of ( dim V dim X ) 1 q (for 1 p + 1 q = 1).

However, arriving at such a functional form from the derivations in #OTHEREFR requires multiple non-trivial steps not carried out in those works.

(iv) #OTHEREFR predicts that adversarial success is a function of 2,p) .

The predictions of Items (i) to (iii) are all consistent with our experimental results, suggesting that the situation where adversarial examples are constrained to a subspace of dimension d is effectively equivalent to the unconstrained situation where data is found in an ambient space of dimension d.

#OTHEREFR The dependence of adversarial success on ( dim V dim X ) 1 q can be derived from the simplest possible toy model, namely binary linear classification. Let X = R n and suppose

Let V ⊆ R n be a subspace with dim V = d. We will assume there exists an isometry U :

How Many Dimensions Are Required To Find An Adversarial Example?

Abstract: Past work exploring adversarial vulnerability have focused on situations where an adversary can perturb all dimensions of model input. On the other hand, a range of recent works consider the case where either (i) an adversary can perturb a limited number of input parameters or (ii) a subset of modalities in a multimodal problem. In both of these cases, adversarial examples are effectively constrained to a subspace V in the ambient input space X. Motivated by this, in this work we investigate how adversarial vulnerability depends on dim(V). In particular, we show that the adversarial success of standard PGD attacks with p norm constraints behaves like a monotonically increasing function of (dim(V) dim X) 1 q where is the perturbation budget and 1 p + 1 q = 1, provided p > 1 (the case p = 1 presents additional subtleties which we analyze in some detail). This functional form can be easily derived from a simple toy linear model, and as such our results land further credence to arguments that adversarial examples are endemic to locally linear models on high dimensional spaces.

Are Adversarial Examples Inevitable?

Abstract: A wide range of defenses have been proposed to harden neural networks against adversarial attacks. However, a pattern has emerged in which the majority of adversarial defenses are quickly broken by new attacks. Given the lack of success at generating robust defenses, we are led to ask a fundamental question: Are adversarial attacks inevitable? This paper analyzes adversarial examples from a theoretical perspective, and identifies fundamental bounds on the susceptibility of a classifier to adversarial attacks. We show that, for certain classes of problems, adversarial examples are inescapable. Using experiments, we explore the implications of theoretical guarantees for real-world problems and discuss how factors such as dimensionality and image complexity limit a classifier's robustness against adversarial examples.


Attacks Against Models For Source Code Analysis

The general white-box attack of #REFR iteratively substitutes a target variable name in all of its occurrences with an alternative name until a misclassification occurs.

This section covers some prominent attacks against models that work on source code.

The techniques from these and other works have limited applicability to binary similarity, as their perturbations may not survive compilation (e.g., variable renaming) or result in marginal differences in compiled code (e.g., turning a while-loop into a for-loop).

The attack against plagiarism detection from #OTHEREFR uses genetic programming to augment a program with code lines picked from a pool and validated for program equivalence by checking that an optimizer compiler removes them.

The attack against clone detection from #OTHEREFR combines several semantics-preserving transformations of source code using different optimization heuristic strategies.

Adversarial Attacks Against Binary Similarity Systems

Abstract: In recent years, binary analysis gained traction as a fundamental approach to inspect software and guarantee its security. Due to the exponential increase of devices running software, much research is now moving towards new autonomous solutions based on deep learning models, as they have been showing state-of-the-art performances in solving binary analysis problems. One of the hot topics in this context is binary similarity, which consists in determining if two functions in assembly code are compiled from the same source code. However, it is unclear how deep learning models for binary similarity behave in an adversarial context. In this paper, we study the resilience of binary similarity models against adversarial examples, showing that they are susceptible to both targeted and untargeted attacks (w.r.t. similarity goals) performed by black-box and white-box attackers. In more detail, we extensively test three current state-of-the-art solutions for binary similarity against two black-box greedy attacks, including a new technique that we call Spatial Greedy, and one white-box attack in which we repurpose a gradient-guided strategy used in attacks to image classifiers.

Adversarial Examples For Models Of Code

Abstract: Neural models of code have shown impressive performance for tasks such as predicting method names and identifying certain kinds of bugs. In this paper, we show that these models are vulnerable to adversarial examples, and introduce a novel approach for attacking trained models of code with adversarial examples. The main idea is to force a given trained model to make an incorrect prediction as specified by the adversary by introducing small perturbations that do not change the program's semantics. To find such perturbations, we present a new technique for Discrete Adversarial Manipulation of Programs (DAMP). DAMP works by deriving the desired prediction with respect to the model's inputs while holding the model weights constant and following the gradients to slightly modify the code. ::: To defend a model against such attacks, we propose placing a defensive model (Anti-DAMP) in front of it. Anti-DAMP detects unlikely mutations and masks them before feeding the input to the downstream model. ::: We show that our DAMP attack is effective across three neural architectures: code2vec, GGNN, and GNN-FiLM, in both Java and C#. We show that DAMP has up to 89% success rate in changing a prediction to the adversary's choice ("targeted attack"), and a success rate of up to 94% in changing a given prediction to any incorrect prediction ("non-targeted attack"). By using Anti-DAMP, the success rate of the attack drops drastically for both targeted and non-targeted attacks, with a minor penalty of 2% relative degradation in accuracy while not performing under attack.


Our main contribution here is to show that a nice ordering for the disks exists to make the co-2-subdivision approach to work, which at the same time improves the inapproximability factor of #REFR .

We now create a rectangle R b to represent b.

We place the top-left corner of R b near the intersection point of the boundaries of the disks for t i , t i+1 such that R b does not intersect these disks but intersects all other type-t disks.

We place the bottom-right corner of R b near the circular segment determined by the disk for s j on I t such that R b does not intersect the disk for s j but intersects all other type-s disks. We refer to #OTHEREFR for a formal reduction. The reason that Bonnet et al.

#OTHEREFR thought such a reduction with a co-2-subdivision approach is unlikely is that without the pair-ordered labeling, the rectangle for R b may require to avoid three arbitrary disks.

However, such realization may not be possible for all the rectangles if the disks are not carefully ordered.

Finding A Maximum Clique In A Disk Graph

Abstract: A disk graph is an intersection graph of disks in the Euclidean plane, where the disks correspond to the vertices of the graph and a pair of vertices are adjacent if and only if their corresponding disks intersect. The problem of determining the time complexity of computing a maximum clique in a disk graph is a long-standing open question that has been very well studied in the literature. The problem is known to be open even when the radii of all the disks are in the interval [1, (1 + ε)], where ε > 0. If all the disks are unit disks then there exists an O(n 3 log n)-time algorithm to compute a maximum clique, which is the best-known running time for over a decade. Although the problem of computing a maximum clique in a disk graph remains open, it is known to be APX-hard for the intersection graphs of many other convex objects such as intersection graphs of ellipses, triangles, and a combination of unit disks and axis-parallel rectangles. Here we obtain the following results.-We give an algorithm to compute a maximum clique in a unit disk graph in O(n 2.5 log n)-time, which improves the previously best known running time of O(n 3 log n) [Eppstein '09].-We extend a widely used 'co-2-subdivision approach' to prove that computing a maximum clique in a combination of unit disks and axis-parallel rectangles is NP-hard to approximate within 4448/4449 ≈ 0.9997. The use of a 'co-2-subdivision approach' was previously thought to be unlikely in this setting [Bonnet et al. '20]. Our result improves the previously known inapproximability factor of 7633010347/7633010348 ≈ 0.9999.-We show that the parameter minimum lens width of the disk arrangement may be used to make progress in the case when disk radii are in [1, (1 + ε)]. For example, if the minimum lens width is at least 0.265 and ε ≤ 0.0001, which still allows for non-Helly triples in the arrangement, then one can find a maximum clique in polynomial time.

Maximum Clique In Disk-Like Intersection Graphs

Abstract: We study the complexity of Maximum Clique in intersection graphs of convex objects in the plane. On the algorithmic side, we extend the polynomial-time algorithm for unit disks [Clark '90, Raghavan and Spinrad '03] to translates of any fixed convex set. We also generalize the efficient polynomial-time approximation scheme (EPTAS) and subexponential algorithm for disks [Bonnet et al. '18, Bonamy et al. '18] to homothets of a fixed centrally symmetric convex set. The main open question on that topic is the complexity of Maximum Clique in disk graphs. It is not known whether this problem is NP-hard. We observe that, so far, all the hardness proofs for Maximum Clique in intersection graph classes $\mathcal I$ follow the same road. They show that, for every graph $G$ of a large-enough class $\mathcal C$, the complement of an even subdivision of $G$ belongs to the intersection class $\mathcal I$. Then they conclude invoking the hardness of Maximum Independent Set on the class $\mathcal C$, and the fact that the even subdivision preserves that hardness. However there is a strong evidence that this approach cannot work for disk graphs [Bonnet et al. '18]. We suggest a new approach, based on a problem that we dub Max Interval Permutation Avoidance, which we prove unlikely to have a subexponential-time approximation scheme. We transfer that hardness to Maximum Clique in intersection graphs of objects which can be either half-planes (or unit disks) or axis-parallel rectangles. That problem is not amenable to the previous approach. We hope that a scaled down (merely NP-hard) variant of Max Interval Permutation Avoidance could help making progress on the disk case, for instance by showing the NP-hardness for (convex) pseudo-disks.