Since the SAC algorithm is very sensitive to hyperparameters #REFR , the following study is performed based on the DDPG and TD3 algorithms.

In addition, the TRPO and PPO algorithms limit the policy updates based on policy similarity, slowing and stabilizing policy updates.

On the other hand, there is currently a lack of quantitative evaluation of power flow nonconvergence, e.g., the reward in (18) is always equal to -1000 when the power flow does not converge, which cannot provide guiding information for policy updates.

In contrast, off-policy DRL algorithms store historical experience, which has a chance to influence the training of the current policy before being removed from the replay buffer.

Therefore, as the exploration continues, the number of experiences corresponding to convergent power flows in the replay buffer increases and the agent can easily learn how to generate actions that lead to convergent power flows.

Therefore, off-policy DRL algorithms are selected to solve the TSC-OPF problem in this paper.

The application of the SAC algorithm will be one of the future research directions.

In the DDPG and TD3 algorithms, during the training process, exploration noise is added to the actions generated by the agent, as shown in #OTHEREFR where o denotes the observation vector, a and a are vectors representing the upper and lower bounds for the action respectively, denotes the exploration rate, σ represents the noise vector that follows a normal distribution with a mean of zero and a variance of , and the clip function restricts the action with exploration noise to be within its upper and lower bounds.

The exploration noise is added to explore the action space and obtain transitions The agent structure of the DDPG algorithm is shown in Fig.

3 (a) , which consists of two sets of actorcritic formats.

The dotted lines mean that the modules and vectors are obsolete, which will be explained later.

On Fast-Converged Reinforcement Learning For Optimal Dispatch Of Large-Scale Power Systems Under Transient Security Constraints

Abstract: Deep Reinforcement Learning (DRL)-based power system optimal dispatch, which is often modeled as Transient Security-Constrained Optimal Power Flow (TSC-OPF), trains efficient dispatching agents that can adapt to different scenarios and provide control strategies quickly. However, three typical issues seriously affect the training efficiency and the performance of the dispatch agent, namely, the difficulty of quantifying the transient instability level, the high dimensionality of the state space and action space, and the frequent generation of actions that correspond to non-convergent power flows during the early training stage. To address these issues, a fast-converged DRL method for TSC-OPF is proposed in this paper. Firstly, a transient security constraint transcription method based on the simulation time duration of instability is proposed to quantify the instability level. Secondly, a general method for Markov decision process modeling of TSC-OPF is proposed to decrease the dimensionality of the observation space. Finally, two general improvement techniques for off-policy DRL algorithms are proposed. A warm-up training technique is introduced to improve the efficiency of agents learning how to generate actions that lead to convergent power flows. A parallel exploration technique is adopted to improve the efficiency of agents exploring the action space. Based on the above studies, environments for TSC-OPF with the objectives of minimizing generation cost and minimizing control cost are constructed and dispatch agents are built and trained. The proposed method is tested in the IEEE 39-bus system and a practical 710-bus regional power grid. Test results show that the training process converges rapidly, the success rate of dispatch in both cases exceeds 99.70 percent, and the decision-making costs very little time, which verifies the effectiveness and efficiency of the proposed method.

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning With A Stochastic Actor

Abstract: Model-free deep reinforcement learning (RL) algorithms have been demonstrated on a range of challenging decision making and control tasks. However, these methods typically suffer from two major challenges: very high sample complexity and brittle convergence properties, which necessitate meticulous hyperparameter tuning. Both of these challenges severely limit the applicability of such methods to complex, real-world domains. In this paper, we propose soft actor-critic, an off-policy actor-critic deep RL algorithm based on the maximum entropy reinforcement learning framework. In this framework, the actor aims to maximize expected reward while also maximizing entropy - that is, succeed at the task while acting as randomly as possible. Prior deep RL methods based on this framework have been formulated as Q-learning methods. By combining off-policy updates with a stable stochastic actor-critic formulation, our method achieves state-of-the-art performance on a range of continuous control benchmark tasks, outperforming prior on-policy and off-policy methods. Furthermore, we demonstrate that, in contrast to other off-policy algorithms, our approach is very stable, achieving very similar performance across different random seeds.


As reported in #REFR , an intuitive interpretation of (4) is imagining the evolution of the vector field U(x, t) over time as a "smearing out" of the initial vector field F(x).

U : M × R + 0 → T M and U(•, t) ∈ L 2 (T M) ∀t ∈ R +

0 ; we denote the initial condition condition with U(x, 0) = F(x).

In this interpretation, the role of the Connection Laplacian can be understood as a means to diffuse vectors from one tangent space to another (indeed, in the "flat" case it is sufficient to independently diffuse each scalar component, however, this approach fails for curved space). The solution of (4) is

which provides a way to construct tangent bundle convolution, as explained in the following section.

Tangent Bundle Convolutional Learning: From Manifolds To Cellular Sheaves And Back

Abstract: In this work we introduce a convolution operation over the tangent bundle of Riemann manifolds in terms of exponentials of the Connection Laplacian operator. We define tangent bundle filters and tangent bundle neural networks (TNNs) based on this convolution operation, which are novel continuous architectures operating on tangent bundle signals, i.e. vector fields over the manifolds. Tangent bundle filters admit a spectral representation that generalizes the ones of scalar manifold filters, graph filters and standard convolutional filters in continuous time. We then introduce a discretization procedure, both in the space and time domains, to make TNNs implementable, showing that their discrete counterpart is a novel principled variant of the very recently introduced sheaf neural networks. We formally prove that this discretized architecture converges to the underlying continuous TNN. Finally, we numerically evaluate the effectiveness of the proposed architecture on various learning tasks, both on synthetic and real data.

The Vector Heat Method

Abstract: This paper describes a method for efficiently computing parallel transport of tangent vectors on curved surfaces, or more generally, any vector-valued data on a curved manifold. More precisely, it extends a vector field defined over any region to the rest of the domain via parallel transport along shortest geodesics. This basic operation enables fast, robust algorithms for extrapolating level set velocities, inverting the exponential map, computing geometric medians and Karcher/Fr\'echet means of arbitrary distributions, constructing centroidal Voronoi diagrams, and finding consistently ordered landmarks. Rather than evaluate parallel transport by explicitly tracing geodesics, we show that it can be computed via a short-time heat flow involving the connection Laplacian. As a result, transport can be achieved by solving three prefactored linear systems, each akin to a standard Poisson problem. Moreover, to implement the method we need only a discrete connection Laplacian, which we describe for a variety of geometric data structures (point clouds, polygon meshes, etc.). We also study the numerical behavior of our method, showing empirically that it converges under refinement, and augment the construction of intrinsic Delaunay triangulations (iDT) so that they can be used in the context of tangent vector field processing.


The two-stage HOI detection framework detects human and object with an off-the-shelf detector #REFR and then classifies the interaction label for each human-object pair.

After the appearance of HO-RCNN #OTHEREFR , which is a widely used multi-stream framework, many recent studies use a variety of additional information to get richer contextual features for the interaction classifier, such as spatial features #OTHEREFR , pose features #OTHEREFR , and linguistic features #OTHEREFR .

Several studies #OTHEREFR attempted to encode global contextual information using a message passing mechanism in a graph structure. Figure 2 . Overview of our ViPLO network.

We first detect human and objects in a given image with Faster-RCNN #OTHEREFR , then estimate each human pose with an off-the-shelf pose estimator.

Then, we extract features for each human and object using a ViT backbone and our novel MOA module.

We also extract local features for each human with the estimated pose and ROIAlign #OTHEREFR .

Viplo: Vision Transformer Based Pose-Conditioned Self-Loop Graph For Human-Object Interaction Detection

Abstract: Human-Object Interaction (HOI) detection, which localizes and infers relationships between human and objects, plays an important role in scene understanding. Although two-stage HOI detectors have advantages of high efficiency in training and inference, they suffer from lower performance than one-stage methods due to the old backbone networks and the lack of considerations for the HOI perception process of humans in the interaction classifiers. In this paper, we propose Vision Transformer based Pose-Conditioned Self-Loop Graph (ViPLO) to resolve these problems. First, we propose a novel feature extraction method suitable for the Vision Transformer backbone, called masking with overlapped area (MOA) module. The MOA module utilizes the overlapped area between each patch and the given region in the attention function, which addresses the quantization problem when using the Vision Transformer backbone. In addition, we design a graph with a pose-conditioned self-loop structure, which updates the human node encoding with local features of human joints. This allows the classifier to focus on specific human joints to effectively identify the type of interaction, which is motivated by the human perception process for HOI. As a result, ViPLO achieves the state-of-the-art results on two public benchmarks, especially obtaining a +2.

Faster R-Cnn: Towards Real-Time Object Detection With Region Proposal Networks

Abstract: State-of-the-art object detection networks depend on region proposal algorithms to hypothesize object locations. Advances like SPPnet [1] and Fast R-CNN [2] have reduced the running time of these detection networks, exposing region proposal computation as a bottleneck. In this work, we introduce a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals. An RPN is a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position. The RPN is trained end-to-end to generate high-quality region proposals, which are used by Fast R-CNN for detection. We further merge RPN and Fast R-CNN into a single network by sharing their convolutional features—using the recently popular terminology of neural networks with ’attention’ mechanisms, the RPN component tells the unified network where to look. For the very deep VGG-16 model [3] , our detection system has a frame rate of 5 fps ( including all steps ) on a GPU, while achieving state-of-the-art object detection accuracy on PASCAL VOC 2007, 2012, and MS COCO datasets with only 300 proposals per image. In ILSVRC and COCO 2015 competitions, Faster R-CNN and RPN are the foundations of the 1st-place winning entries in several tracks. Code has been made publicly available.


For general LDP randomizers, the #REFR shows the input-independent part is sampled by each user with probability at least e − 0 (they term it as total variation similarity).

In this model, a shuffler breaks the association between messages and their owners' identities, which allows for lower levels of randomization in the local model.

An important aspect of the shuffle model is the analysis of privacy amplification guarantee for shuffled messages, with tighter guarantees leading to better trade-offs between privacy and utility.

Privacy amplification of LDP randomizers.

The seminal work #OTHEREFR utilizes the privacy amplification via subsampling #OTHEREFR to analyze the privacy amplification of shuffling, and shows n shuffled messages satisfies ( 0 144 log(1/δ)/n, δ)-DP.

Latterly, the privacy blanket #OTHEREFR proposes extracting an input-independent part from the output distribution, to work as a "blanket" to amplify privacy.

For specific LDP randomizers (e.g., Laplace #OTHEREFR and generalized randomized response #OTHEREFR ), the total variation similarity can be larger.

We note that our framework also utilizes the total variation information about output distributions.

Compared to the total variation similarity that must be shared by all output distributions, we care about the pairwise total variation bound of output distributions, which is at most e 0 −1 e 0 +1 #OTHEREFR Theorem 2.4] .

Recently, the works #OTHEREFR and #OTHEREFR decompose output distributions into mixture distributions with 3 options, and interprets messages from other users as clones of victim user.

They show the shuffled messages can be deemed as post-processed information on the multiplicity of the 3 options, thus reduces the analyses of privacy amplification on shuffled messages to the indistinguishability of the multiplicities. This clone reduction is near-optimal w.r.t. the dependence on 0 for general LDP randomizers.

Privacy Amplification Via Shuffling: Unified, Simplified, And Tightened

Abstract: In decentralized settings, the shuffle model of differential privacy has emerged as a promising alternative to the classical local model. Analyzing privacy amplification via shuffling is a critical component in both single-message and multi-message shuffle protocols. However, current methods used in these two areas are distinct and specific, making them less convenient for protocol designers and practitioners. In this work, we introduce variation-ratio reduction as a unified framework for privacy amplification analyses in the shuffle model. This framework utilizes total variation bounds of local messages and probability ratio bounds of other users' blanket messages, converting them to indistinguishable levels. Our results indicate that the framework yields tighter bounds for both singlemessage and multi-message encoders (e.g., with local DP, local metric DP, or general multi-message randomizers). Specifically, for a broad range of local randomizers having extremal probability design, our amplification bounds are precisely tight. We also demonstrate that variation-ratio reduction is well-suited for parallel composition in the shuffle model and results in stricter privacy accounting for common sampling-based local randomizers. Our experimental findings show that, compared to existing amplification bounds, our numerical amplification bounds can save up to 30% of the budget for singlemessage protocols, 75% of the budget for multi-message protocols, and 75%-95% of the budget for parallel composition. Additionally, our implementation for numerical amplification bounds has onlyÕ(n) complexity and is highly efficient in practice, taking just 2 minutes for n = 10 8 users. The code for our implementation can be found at https://github.com/wangsw/PrivacyAmplification.

The Privacy Blanket Of The Shuffle Model

Abstract: This work studies differential privacy in the context of the recently proposed shuffle model. Unlike in the local model, where the server collecting privatized data from users can track back an input to a specific user, in the shuffle model users submit their privatized inputs to a server anonymously. This setup yields a trust model which sits in between the classical curator and local models for differential privacy. The shuffle model is the core idea in the Encode, Shuffle, Analyze (ESA) model introduced by Bittau et al. (SOPS 2017). Recent work by Cheu et al. (EUROCRYPT 2019) analyzes the differential privacy properties of the shuffle model and shows that in some cases shuffled protocols provide strictly better accuracy than local protocols. Additionally, Erlingsson et al. (SODA 2019) provide a privacy amplification bound quantifying the level of curator differential privacy achieved by the shuffle model in terms of the local differential privacy of the randomizer used by each user. In this context, we make three contributions. First, we provide an optimal single message protocol for summation of real numbers in the shuffle model. Our protocol is very simple and has better accuracy and communication than the protocols for this same problem proposed by Cheu et al. Optimality of this protocol follows from our second contribution, a new lower bound for the accuracy of private protocols for summation of real numbers in the shuffle model. The third contribution is a new amplification bound for analyzing the privacy of protocols in the shuffle model in terms of the privacy provided by the corresponding local randomizer. Our amplification bound generalizes the results by Erlingsson et al. to a wider range of parameters, and provides a whole family of methods to analyze privacy amplification in the shuffle model.


It has been observed in verification literature that neural networks often fail to satisfy logical constraints #REFR .

Analysis of properties of loss functions, especially smoothness #OTHEREFR or bilateral properties #OTHEREFR , is a prominent field #OTHEREFR .

One of LDL's achievements is to expose trade-offs between satisfying desired geometric and logic properties of a loss functions.

In the future, we plan to explore further technical ideas from this field.

While this work does not attempt to verify neural networks, we draw our motivation from this area of research.

One of proposed solutions is training the NN to satisfy a constraint prior to verifying them #OTHEREFR 44] , referred to as continuous verification #OTHEREFR . LDL fits into this trend.

Indeed, the tool Vehicle that implements LDL is also built to work with SMT-solvers and NN verifiers #OTHEREFR .

Logics for Uncertainty and Probabilistic Logics.

LDLs have a strong connection to fuzzy logic #OTHEREFR , as we have shown.

Via the use of probability distributions and expectations, we draw our connection to Probabilistic Prolog and similar languages #OTHEREFR .

Logic Of Differentiable Logics: Towards A Uniform Semantics Of Dl

Abstract: Differentiable logics (DL) have recently been proposed as a method of training neural networks to satisfy logical specifications. A DL consists of a syntax in which specifications are stated and an interpretation function that translates expressions in the syntax into loss functions. These loss functions can then be used during training with standard gradient descent algorithms. The variety of existing DLs and the differing levels of formality with which they are treated makes a systematic comparative study of their properties and implementations difficult. This paper remedies this problem by suggesting a metalanguage for defining DLs that we call the Logic of Differentiable Logics, or LDL. Syntactically, it generalises the syntax of existing DLs to FOL, and for the first time introduces the formalism for reasoning about vectors and learners. Semantically, it introduces a general interpretation function that can be instantiated to define loss functions arising from different existing DLs. We use LDL to establish several theoretical properties of existing DLs, and to conduct their empirical study in neural network verification.

Efficient Formal Safety Analysis Of Neural Networks

Abstract: Neural networks are increasingly deployed in real-world safety-critical domains such as autonomous driving, aircraft collision avoidance, and malware detection. However, these networks have been shown to often mispredict on inputs with minor adversarial or even accidental perturbations. Consequences of such errors can be disastrous and even potentially fatal as shown by the recent Tesla autopilot crash. Thus, there is an urgent need for formal analysis systems that can rigorously check neural networks for violations of different safety properties such as robustness against adversarial perturbations within a certain $L$-norm of a given image. An effective safety analysis system for a neural network must be able to either ensure that a safety property is satisfied by the network or find a counterexample, i.e., an input for which the network will violate the property. Unfortunately, most existing techniques for performing such analysis struggle to scale beyond very small networks and the ones that can scale to larger networks suffer from high false positives and cannot produce concrete counterexamples in case of a property violation. In this paper, we present a new efficient approach for rigorously checking different safety properties of neural networks that significantly outperforms existing approaches by multiple orders of magnitude. Our approach can check different safety properties and find concrete counterexamples for networks that are 10$\times$ larger than the ones supported by existing analysis techniques. We believe that our approach to estimating tight output bounds of a network for a given input range can also help improve the explainability of neural networks and guide the training process of more robust neural networks.


For input corruption, we adopt the masking strategy as in BERT #REFR to randomly mask r% tokens of the input sentences S i , obtaining the corrupted input sentences S i .

, y |Si| }, where y 0 and |S i | denote the beginof-sentence token and the number of tokens, respectively, and utilize the cross-entropy loss, which is widely used in natural language generation problems:

where we implement y 0 as a language-specific token following #OTHEREFR , #OTHEREFR so that the decoder can be aware of which language to be generated.

Data Corruption For vision-to-text, due to (i) the large variations of images and videos caused by different object attributes, occlusion, motion blur and etc #OTHEREFR ; (ii) the great disparities between the vision and the language domains #OTHEREFR , we propose two data corruption strategies to further improve the performance and robustness of our ZeroNLG.

In implementations, we simultaneously consider the input and feature corruptions.

As a result, the language reconstruction process is defined as follows:

For the feature corruption, we propose to add Gaussian noise n ∼ N (0, ) into the text features E m (S i ) (i.e., the coordinates) of input sentences S i , acquiring the corrupted features of input sentences E m (S i ) = E m (S i )+n. Therefore, the reconstruction process is defined as follows:

Through data corruption, we can encourage the model to learn more robust latent representations, achieving strong performances on zero-shot natural language generation.

Zeronlg: Aligning And Autoencoding Domains For Zero-Shot Multimodal And Multilingual Natural Language Generation

Abstract: Natural Language Generation (NLG) accepts input data in the form of images, videos, or text and generates corresponding natural language text as output. Existing NLG methods mainly adopt a supervised approach and rely heavily on coupled data-to-text pairs. However, for many targeted scenarios and for non-English languages, sufficient quantities of labeled data are often not available. As a result, it is necessary to collect and label data-text pairs for training, which is both costly and time-consuming. To relax the dependency on labeled data of downstream tasks, we propose an intuitive and effective zero-shot learning framework, ZeroNLG, which can deal with multiple NLG tasks, including image-to-text (image captioning), video-to-text (video captioning), and text-to-text (neural machine translation), across English, Chinese, German, and French within a unified framework. ZeroNLG does not require any labeled downstream pairs for training. During training, ZeroNLG (i) projects different domains (across modalities and languages) to corresponding coordinates in a shared common latent space; (ii) bridges different domains by aligning their corresponding coordinates in this space; and (iii) builds an unsupervised multilingual auto-encoder to learn to generate text by reconstructing the input text given its coordinate in shared latent space. Consequently, during inference, based on the data-to-text pipeline, ZeroNLG can generate target sentences across different languages given the coordinate of input data in the common space. Within this unified framework, given visual (imaging or video) data as input, ZeroNLG can perform zero-shot visual captioning; given textual sentences as input, ZeroNLG can perform zero-shot machine translation. We present the results of extensive experiments on twelve NLG tasks, showing that, without using any labeled downstream pairs for training, ZeroNLG generates high-quality and "believable" outputs and significantly outperforms existing zero-shot methods.

Bert: Pre-Training Of Deep Bidirectional Transformers For Language Understanding

Abstract: We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications. ::: BERT is conceptually simple and empirically powerful. It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5% (7.7% point absolute improvement), MultiNLI accuracy to 86.7% (4.6% absolute improvement), SQuAD v1.1 question answering Test F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1 (5.1 point absolute improvement).


The Vision Transformer (ViT) based encoder in Pi-MAE is pre-trained on the COCO dataset #REFR to improve performance.

The PiMAE model ( Figure 1 ) consists of three key components: (1) a Vision Transformer-based #OTHEREFR encoder-decoder architecture with a mask layer to prevent trivial solutions while estimating emitters, (2) a Convolutional Neural Network as a prior for PSF estimation #OTHEREFR , and (3) a microscopic imaging process that enforces adherence to the microscopy principle.

Appendix A provides detailed information on the network architecture and the embedding of physical principle.

PiMAE requires only a few raw images for training, which is attributed to the carefully designed loss function.

The loss function consists of two parts: one measures the difference between the raw and the reconstruction images, including the mean of the absolute difference and the multiscale structure similarity; the other part is a constraint on the PSF, including the total variation loss measuring the PSF continuity and the offset distance of the PSF's center of mass. Appendix B contains the expressions for the loss functions.

This pre-training relies on the selfsupervised learning of a masked autoencoder, but does not incorporate any physical information (detailed in Appendix C).

After pre-training, PiMAE loads the trained encoder parameters and undergoes self-supervised training using raw microscopic images.

The input image size is 144 pixels, and we use the RAdam optimizer #OTHEREFR with a learning rate of 1e −4 and a batch size of 18. The training runs for 5e 4 steps.

Within PiMAE, the convolutional neural network, depicted in Figure 1 , is initialized randomly and takes a fixed random vector as input, outputting the predicted PSF. Relevant details can be found in Appendix A.

As Pi-MAE undergoes self-supervised training, the CNN's predicted PSF continually becomes more accurate, moving closer to the true PSF as shown in Figure 2 . The experimental setup is shown in Figure 3 .

Learning Imaging Mechanism Directly From Optical Microscopy Observations

Abstract: Optical microscopy image plays an important role in scientific research through the direct visualization of the nanoworld, where the imaging mechanism is described as the convolution of the point spread function (PSF) and emitters. Based on a priori knowledge of the PSF or equivalent PSF, it is possible to achieve more precise exploration of the nanoworld. However, it is an outstanding challenge to directly extract the PSF from microscopy images. Here, with the help of self-supervised learning, we propose a physics-informed masked autoencoder (PiMAE) that enables a learnable estimation of the PSF and emitters directly from the raw microscopy images. We demonstrate our method in synthetic data and real-world experiments with significant accuracy and noise robustness. PiMAE outperforms DeepSTORM and the Richardson-Lucy algorithm in synthetic data tasks with an average improvement of 19.6% and 50.7% (35 tasks), respectively, as measured by the normalized root mean square error (NRMSE) metric. This is achieved without prior knowledge of the PSF, in contrast to the supervised approach used by DeepSTORM and the known PSF assumption in the Richardson-Lucy algorithm. Our method, PiMAE, provides a feasible scheme for achieving the hidden imaging mechanism in optical microscopy and has the potential to learn hidden mechanisms in many more systems.

Microsoft Coco: Common Objects In Context

Abstract: We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding. This is achieved by gathering images of complex everyday scenes containing common objects in their natural context. Objects are labeled using per-instance segmentations to aid in precise object localization. Our dataset contains photos of 91 objects types that would be easily recognizable by a 4 year old. With a total of 2.5 million labeled instances in 328k images, the creation of our dataset drew upon extensive crowd worker involvement via novel user interfaces for category detection, instance spotting and instance segmentation. We present a detailed statistical analysis of the dataset in comparison to PASCAL, ImageNet, and SUN. Finally, we provide baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model.


DFANet #REFR adopted a lightweight backbone to reduce computational cost and designed crosslevel aggregation for feature refinement.

As a fundamental task of scene understanding, semantic segmentation has been an active research area for many years #OTHEREFR , which also attracts considerable attention in the study of deep neural networks, e.g., FCN #OTHEREFR , DeepLabs #OTHEREFR and PSPNet #OTHEREFR .

In order to obtain accurate results in real-time applications, several methods have been proposed to improve the efficiency of semantic segmentation, which we summarize as follows.

Many compact architectures have been proposed for efficient image segmentation.

DFNet #OTHEREFR utilized a partial order pruning algorithm to search segmentation models for a good trade-off between speed and accuracy.

ICNet #OTHEREFR used a cascade fusion module and transformed part of the computation from high-resolution to low-resolution. Wang et al. #OTHEREFR designed superresolution learning to improve image segmentation performance.

BiSeNets #OTHEREFR used two-stream paths for lowlevel details and high-level context information, respectively.

ESPNet #OTHEREFR used an efficient spatial pyramid to accelerate the convolution computation.

These efficient backbone networks reduce the computational burden of singleimage segmentation, and can be applied to temporal or spatial frameworks in VSS.

Efficient Semantic Segmentation By Altering Resolutions For Compressed Videos

Abstract: Video semantic segmentation (VSS) is a computationally expensive task due to the per-frame prediction for videos of high frame rates. In recent work, compact models or adaptive network strategies have been proposed for efficient VSS. However, they did not consider a crucial factor that affects the computational cost from the input side: the input resolution. In this paper, we propose an altering resolution framework called AR-Seg for compressed videos to achieve efficient VSS. AR-Seg aims to reduce the computational cost by using low resolution for non-keyframes. To prevent the performance degradation caused by downsampling, we design a Cross Resolution Feature Fusion (CR-eFF) module, and supervise it with a novel Feature Similarity Training (FST) strategy. Specifically, CReFF first makes use of motion vectors stored in a compressed video to warp features from high-resolution keyframes to low-resolution non-keyframes for better spatial alignment, and then selectively aggregates the warped features with local attention mechanism. Furthermore, the proposed FST supervises the aggregated features with high-resolution features through an explicit similarity loss and an implicit constraint from the shared decoding layer. Extensive experiments on CamVid and Cityscapes show that AR-Seg achieves state-of-the-art performance and is compatible with different segmentation backbones. On CamVid, AR-Seg saves 67% computational cost (measured in GFLOPs) with the PSPNet18 backbone while maintaining high segmentation accuracy.

Dfanet: Deep Feature Aggregation For Real-Time Semantic Segmentation

Abstract: This paper introduces an extremely efficient CNN architecture named DFANet for semantic segmentation under resource constraints. Our proposed network starts from a single lightweight backbone and aggregates discriminative features through sub-network and sub-stage cascade respectively. Based on the multi-scale feature propagation, DFANet substantially reduces the number of parameters, but still obtains sufficient receptive field and enhances the model learning ability, which strikes a balance between the speed and segmentation performance. Experiments on Cityscapes and CamVid datasets demonstrate the superior performance of DFANet with 8$\times$ less FLOPs and 2$\times$ faster than the existing state-of-the-art real-time semantic segmentation methods while providing comparable accuracy. Specifically, it achieves 70.3\% Mean IOU on the Cityscapes test dataset with only 1.7 GFLOPs and a speed of 160 FPS on one NVIDIA Titan X card, and 71.3\% Mean IOU with 3.4 GFLOPs while inferring on a higher resolution image.


If "people" exists, we may select the celebrity model #REFR to further understand whether a celebrity appears and who he/she is.

Recent years have seen significant advancement for computer vision, thanks to improved network architecture #OTHEREFR , large-scale model training #OTHEREFR , and other factors.

However, different vision problems typically require different models, which often require manual selection and composition of individual models for each use case.

For example, when determining if an image contains "people", we may choose the image tagging model #OTHEREFR and check if the predicted tag list contains "people".

One research direction is to combine the vision and language modules as one end-to-end model, such as Flamingo #OTHEREFR , PaLM-E #OTHEREFR , to provide a dialogue-based experience to the end user.

That is, the user can use natural language to interact with the model around the image content.

The vision module encodes vision signals into special text tokens or features that the language module can understand, enabling the system to utilize the language module for understanding user queries and providing responses.

However, these joint finetuning approaches require a large amount of computing resources and annotated data to enable specific capabilities.

In this work, we aim to combine existing individual vision models with the language model in a more flexible manner to tackle complicated visual understanding problems, e.g., the ones illustrated in Figure 1 .

Mm-React : Prompting Chatgpt For Multimodal Reasoning And Action

Abstract: Figure 1. MM-REACT allocates specialized vision experts with ChatGPT to solve challenging visual understanding tasks through multimodal reasoning and action. For example, the system could associate information from multiple uploaded receipts and calculate the total travel cost ("Multi-Image Reasoning"). We only highlight key information here and postpone full MM-REACT responses to Figures 4-14.

Deep Learning Face Attributes In The Wild

Abstract: Predicting face attributes in the wild is challenging due to complex face variations. We propose a novel deep learning framework for attribute prediction in the wild. It cascades two CNNs, LNet and ANet, which are fine-tuned jointly with attribute tags, but pre-trained differently. LNet is pre-trained by massive general object categories for face localization, while ANet is pre-trained by massive face identities for attribute prediction. This framework not only outperforms the state-of-the-art with a large margin, but also reveals valuable facts on learning face representation. (1) It shows how the performances of face localization (LNet) and attribute prediction (ANet) can be improved by different pre-training strategies. (2) It reveals that although the filters of LNet are fine-tuned only with image-level attribute tags, their response maps over entire images have strong indication of face locations. This fact enables training LNet for face localization with only image-level annotations, but without face bounding boxes or landmarks, which are required by all attribute recognition works. (3) It also demonstrates that the high-level hidden neurons of ANet automatically discover semantic concepts after pre-training with massive face identities, and such concepts are significantly enriched after fine-tuning with attribute tags. Each attribute can be well explained with a sparse linear combination of these concepts.


Related Work 5.1 Knowledge Embedding Methods

KBAT #REFR employ Graph Neural Networks (GNN) as the encoder to aggregate multi-hop neighborhood information.

Knowledge embedding methods have been widely used in graph representation learning tasks and have achieved great success on knowledge base completion (a.k.a link prediction).

Translationbased methods aim at finding the transformation relationships from source to target.

TransE #OTHEREFR , the most representative translationbased model, projects entities and relations into a unified vector space and minimizes the energy function of triples. Following this route, many translation-based methods have emerged. TransH #OTHEREFR formulates the translating process on relation-specific hyperplanes. TransR #OTHEREFR projects entities and relations into separate spaces.

Recently, some neural network methods have shown promising results in this task.

ConvE #OTHEREFR and ConvKB #OTHEREFR utilize Convolutional Neural Network (CNN) to increase parameter interaction between entities and relations.

Imf: Interactive Multimodal Fusion Model For Link Prediction

Abstract: Link prediction aims to identify potential missing triples in knowledge graphs. To get better results, some recent studies have introduced multimodal information to link prediction. However, these methods utilize multimodal information separately and neglect the complicated interaction between different modalities. In this paper, we aim at better modeling the inter-modality information and thus introduce a novel Interactive Multimodal Fusion (IMF) model to integrate knowledge from different modalities. To this end, we propose a two-stage multimodal fusion framework to preserve modality-specific knowledge as well as take advantage of the complementarity between different modalities. Instead of directly projecting different modalities into a unified space, our multimodal fusion module limits the representations of different modalities independent while leverages bilinear pooling for fusion and incorporates contrastive learning as additional constraints. Furthermore, the decision fusion module delivers the learned weighted average over the predictions of all modalities to better incorporate the complementarity of different modalities. Our approach has been demonstrated to be effective through empirical evaluations on several real-world datasets. The implementation code

Learning Attention-Based Embeddings For Relation Prediction In Knowledge Graphs

Abstract: The recent proliferation of knowledge graphs (KGs) coupled with incomplete or partial information, in the form of missing relations (links) between entities, has fueled a lot of research on knowledge base completion (also known as relation prediction). Several recent works suggest that convolutional neural network (CNN) based models generate richer and more expressive feature embeddings and hence also perform well on relation prediction. However, we observe that these KG embeddings treat triples independently and thus fail to cover the complex and hidden information that is inherently implicit in the local neighborhood surrounding a triple. To this effect, our paper proposes a novel attention based feature embedding that captures both entity and relation features in any given entity's neighborhood. Additionally, we also encapsulate relation clusters and multihop relations in our model. Our empirical study offers insights into the efficacy of our attention based model and we show marked performance gains in comparison to state of the art methods on all datasets.


Additionally, as distributed ledgers can be publicly accessed, concerns about privacy and anonymity are duly noted #REFR .

Public blockchains are also termed as permissionless blockchains, while private and consortium blockchains are permissioned.

• Public Blockchains: Public blockchains, with Bitcoin and Ethereum as two typical examples, are decentralized, open to the public, and self-governed.

Anyone can join or leave a public blockchain at any time, and all can participate in the consensus process as validators.

Public blockchains often use incentives, such as transaction fees and mining rewards, to encourage nodes to maintain the normal operations of the network.

However, they face challenges from malicious nodes that can launch token stealing #OTHEREFR , selfish mining #OTHEREFR , Sybil #OTHEREFR , eclipse #OTHEREFR , and various other attacks.

Power consumption is another significant issue in public proof-of-work-based blockchains, especially when they are deployed in a large scale.

• Private Blockchains: Private blockchains, with Multichain #OTHEREFR , Monax [136] , and Blockstack [1] as typical examples, are used and governed by single organizations.

Instead of being open to anyone, the access to a private blockchain is restricted and requires a verified invitation.

As a result, private blockchains are not decentralized, but permission-based and closed, which makes it easier to manage them and provides better privacy, but sacrifices decentralization and openness.

The operator of a private blockchain has the ability to override, roll back, delete, and edit blocks, which undermines blockchain's trustless property.

Exploring Blockchain Technology Through A Modular Lens: A Survey

Abstract: Blockchain has attracted significant attention in recent years due to its potential to revolutionize various industries by providing trustlessness. To comprehensively examine blockchain systems, this article presents both a macro-level overview on the most popular blockchain systems, and a micro-level analysis on a general blockchain framework and its crucial components. The macro-level exploration provides a big picture on the endeavors made by blockchain professionals over the years to enhance the blockchain performance while the micro-level investigation details the blockchain building blocks for deep technology comprehension. More specifically, this article introduces a general modular blockchain analytic framework that decomposes a blockchain system into interacting modules and then examines the major modules to cover the essential blockchain components of network, consensus, and distributed ledger at the micro-level. The framework as well as the modular analysis jointly build a foundation for designing scalable, flexible, and application-adaptive blockchains that can meet diverse requirements. Additionally, this article explores popular technologies that can be integrated with blockchain to expand functionality and highlights major challenges. Such a study provides critical insights to overcome the obstacles in designing novel blockchain systems and facilitates the further development of blockchain as a digital infrastructure to service new applications. CCS Concepts: • General and reference → Surveys and overviews.

A Survey On The Security Of Blockchain Systems

Abstract: Since its inception, the blockchain technology has shown promising application prospects. From the initial cryptocurrency to the current smart contract, blockchain has been applied to many fields. Although there are some studies on the security and privacy issues of blockchain, there lacks a systematic examination on the security of blockchain systems. In this paper, we conduct a systematic study on the security threats to blockchain and survey the corresponding real attacks by examining popular blockchain systems. We also review the security enhancement solutions for blockchain, which could be used in the development of various blockchain systems, and suggest some future directions to stir research efforts into this area.


The performance boost remains significant with fine-tuned diffusion models for synthetic data up to a factor of 4 or 5 times the size of the real ImageNet training set, a significant improvement over results reported in #REFR .

Models trained solely on generated samples perform worse than models trained on real data.

Nevertheless, augmenting the real data with data generated from the fine-tuned diffusion model provides a substantial boost in performance across many different classifiers.

creases up to nine times the amount of real data, to a total dataset size of 12M images.

Performance with higher resolution images, however, does not continue to improve with similarly large amounts of generative data augmentation.

Table 4 reports performance as the amount of generated data increased over the same range, up to 9× the amount of real data, at resolutions 256×256 and 1024×1024.

Synthetic Data From Diffusion Models Improves Imagenet Classification

Abstract: Deep generative models are becoming increasingly powerful, now generating diverse high fidelity photo-realistic samples given text prompts. Have they reached the point where models of natural images can be used for generative data augmentation, helping to improve challenging discriminative tasks? We show that large-scale text-toimage diffusion models can be fine-tuned to produce classconditional models with SOTA FID (1.76 at 256×256 resolution) and Inception Score (239 at 256 × 256). The model also yields a new SOTA in Classification Accuracy Scores (64.96 for 256×256 generative samples, improving to 69.24 for 1024×1024 samples). Augmenting the ImageNet training set with samples from the resulting models yields significant improvements in ImageNet classification accuracy over strong ResNet and Vision Transformer baselines.

Classification Accuracy Score For Conditional Generative Models

Abstract: Deep generative models (DGMs) of images are now sufficiently mature that they produce nearly photorealistic samples and obtain scores similar to the data distribution on heuristics such as Frechet Inception Distance (FID). These results, especially on large-scale datasets such as ImageNet, suggest that DGMs are learning the data distribution in a perceptually meaningful space and can be used in downstream tasks. To test this latter hypothesis, we use class-conditional generative models from a number of model classes—variational autoencoders, autoregressive models, and generative adversarial networks (GANs)—to infer the class labels of real data. We perform this inference by training an image classifier using only synthetic data and using the classifier to predict labels on real data. The performance on this task, which we call Classification Accuracy Score (CAS), reveals some surprising results not identified by traditional metrics and constitute our contributions. First, when using a state-of-the-art GAN (BigGAN-deep), Top-1 and Top-5 accuracy decrease by 27.9% and 41.6%, respectively, compared to the original data; and conditional generative models from other model classes, such as Vector-Quantized Variational Autoencoder-2 (VQ-VAE-2) and Hierarchical Autoregressive Models (HAMs), substantially outperform GANs on this benchmark. Second, CAS automatically surfaces particular classes for which generative models failed to capture the data distribution, and were previously unknown in the literature. Third, we find traditional GAN metrics such as Inception Score (IS) and FID neither predictive of CAS nor useful when evaluating non-GAN models. Furthermore, in order to facilitate better diagnoses of generative models, we open-source the proposed metric.


He used this generalized bound to improve a uniform continuity bound for the entanglement of formation originally given by Winter in #REFR .

We have presented a proof of a tight uniform continuity bound for the conditional Shannon entropy.

The bound is independent of the alphabet size of the conditioning system.

However, we have assumed in the proof that the conditioning system has finite support.

We proved a conjectured bound for the conditional von Neumann entropy in the special case where the two systems have the same dimension and the state with lower conditional entropy is diagonal in a maximally entangled basis.

In #OTHEREFR , Wilde generalized our bound Eq. 3.3.7 to quantum-classical states.

In #OTHEREFR , using an approach based on our proof techniques,

Some Problems Concerning Quantum Channels And Entropies

Abstract: Finally, I do not believe it is possible for me to include everybody who ought to be included. To insulate myself, I declare that I am grateful for people.

Tight Uniform Continuity Bounds For Quantum Entropies: Conditional Entropy, Relative Entropy Distance And Energy Constraints

Abstract: We present a bouquet of continuity bounds for quantum entropies, falling broadly into two classes: first, a tight analysis of the Alicki–Fannes continuity bounds for the conditional von Neumann entropy, reaching almost the best possible form that depends only on the system dimension and the trace distance of the states. Almost the same proof can be used to derive similar continuity bounds for the relative entropy distance from a convex set of states or positive operators. As applications, we give new proofs, with tighter bounds, of the asymptotic continuity of the relative entropy of entanglement, ER, and its regularization \({E_R^{\infty}}\), as well as of the entanglement of formation, EF. Using a novel “quantum coupling” of density operators, which may be of independent interest, we extend the latter to an asymptotic continuity bound for the regularized entanglement of formation, aka entanglement cost, \({E_C=E_F^{\infty}}\). Second, we derive analogous continuity bounds for the von Neumann entropy and conditional entropy in infinite dimensional systems under an energy constraint, most importantly systems of multiple quantum harmonic oscillators. While without an energy bound the entropy is discontinuous, it is well-known to be continuous on states of bounded energy. However, a quantitative statement to that effect seems not to have been known. Here, under some regularity assumptions on the Hamiltonian, we find that, quite intuitively, the Gibbs entropy at the given energy roughly takes the role of the Hilbert space dimension in the finite-dimensional Fannes inequality.


Zhang and Wonka (2021) introduce a probabilistic embedding instead of a deterministic one, and also propose a new loss function for the clustering step, with which they achieved good performance on the PartNet dataset #REFR .

(2019b) introduce a linking module between the two decoder branches to exploit synergies between semantic and instance segmentation. Pham et al.

(2019) employ a multi-value conditional random field model to jointly optimise instance and semantic labels. #OTHEREFR

(2019) estimate vectors pointing to the potential instance centers, as in the generalised Hough transform, to support the subsequent clustering step.

In addition to the 3D offset vector, OccuSeg #OTHEREFR also learns occupancy signals, which can guide the subsequent graph-based clustering towards better instance segmentation. #OTHEREFR

(2019) integrate 2D birds-eye-view information into a network for joint 3D semantic and instance segmentation, in order to better exploit global context.

In order to mitigate imbalances in the data, which tend to harm the instance segmentation for rare categories, He et al. (2020) propose a memory-augmented network to memorise representative patterns.

Recently, a number of studies have applied the bottom-up approach to outdoor dataset #OTHEREFR .

In general, these are variants of the two-branch architecture described above. For instance #OTHEREFR

(2020) and used spherical projection to implement a real-time, panoptic segmentation algorithm for the autonomous driving setting, while Panoptic-PolarNet #OTHEREFR used a polar bird's eye view. Their instance branch directly regresses the instance's center.

DS-Net #OTHEREFR utilizes a dynamic shift module that can automatically adjust the kernel function to different point densities and instance sizes.

A Review Of Panoptic Segmentation For Mobile Mapping Point Clouds

Abstract: 3D point cloud panoptic segmentation is the combined task to (i) assign each point to a semantic class and (ii) separate the points in each class into object instances. Recently there has been an increased interest in such comprehensive 3D scene understanding, building on the rapid advances of semantic segmentation due to the advent of deep 3D neural networks. Yet, to date there is very little work about panoptic segmentation of outdoor mobile-mapping data, and no systematic comparisons. The present paper tries to close that gap. It reviews the building blocks needed to assemble a panoptic segmentation pipeline and the related literature. Moreover, a modular pipeline is set up to perform comprehensive, systematic experiments to assess the state of panoptic segmentation in the context of street mapping. As a byproduct, we also provide the first public dataset for that task, by extending the NPM3D dataset to include instance labels.

Partnet: A Large-Scale Benchmark For Fine-Grained And Hierarchical Part-Level 3D Object Understanding

Abstract: We present PartNet: a consistent, large-scale dataset of 3D objects annotated with fine-grained, instance-level, and hierarchical 3D part information. Our dataset consists of 573,585 part instances over 26,671 3D models covering 24 object categories. This dataset enables and serves as a catalyst for many tasks such as shape analysis, dynamic 3D scene modeling and simulation, affordance analysis, and others. Using our dataset, we establish three benchmarking tasks for evaluating 3D part recognition: fine-grained semantic segmentation, hierarchical semantic segmentation, and instance segmentation. We benchmark four state-of-the-art 3D deep learning algorithms for fine-grained semantic segmentation and three baseline methods for hierarchical semantic segmentation. We also propose a baseline method for part instance segmentation and demonstrate its superior performance over existing methods.


We use ResNet50 #REFR as the backbone with an ImageNet-pretrained model from TORCHVISION in all experiments unless specified otherwise. The AdamW [23] optimizer is used.

Note that we replace the official noisy annotation that contains 10% labels generated semi-automatically using #OTHEREFR with the one released by BigDetection #OTHEREFR .

Ob-ject365 is another large-scale dataset, and it contains around 1.72M images with more than 22.8M bounding boxes over 365 categories.

Then we finetune the pre-trained models on the COCO 2017 dataset. Implementation details.

The classification head of METR is the form of the dot-product layer #OTHEREFR and the detailed algorithm will be summarized in supplementary materials.

We conduct our experiments using the PyTorch #OTHEREFR deep learning framework.

The learning rates for the backbone and the transformer are initially set to be 1e−5 and 1e−4, respectively.

The learning rate is dropped by a factor of 10 after 11 epochs for 12 training epochs, and after 40 epochs for 50 training epochs. The weight decay is set to be 1e−4.

We train METR on COCO using 8 Nvidia A100 40G GPUs, and each GPU has a local batch size of 1 image only.

For language embeddings, we select CLIP-B/16 #OTHEREFR text encoder throughout this study.

We adopt most of the default hyper-parameters and data augmentation same as DINO.

Language-Aware Multiple Datasets Detection Pretraining For Detrs

Abstract: Pretraining on large-scale datasets can boost the performance of object detectors while the annotated datasets for object detection are hard to scale up due to the high labor cost. What we possess are numerous isolated filed-specific datasets, thus, it is appealing to jointly pretrain models across aggregation of datasets to enhance data volume and diversity. In this paper, we propose a strong framework for utilizing Multiple datasets to pretrain DETR-like detectors, termed METR, without the need for manual label spaces integration. It converts the typical multi-classification in object detection into binary classification by introducing a pre-trained language model. Specifically, we design a category extraction module for extracting potential categories involved in an image and assign these categories into different queries by language embeddings. Each query is only responsible for predicting a class-specific object. Besides, to adapt our novel detection paradigm, we propose a group bipartite matching strategy that limits the ground truths to match queries assigned to the same category. Extensive experiments demonstrate that METR achieves extraordinary results on either multi-task joint training or the pretrain & finetune paradigm. Notably, our pre-trained models have high flexible transferability and increase the performance upon various DETR-like detectors on COCO val2017 benchmark. Codes will be available after this paper is published.

Deep Residual Learning For Image Recognition

Abstract: Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.


An initial learning rate was 0.001, exponentially decayed with a factor of 0.95 during the distillation step, and varied by SGDR #REFR during the fine-tuning step (T 0 = 2, T mult = 2).

Analogous to the previous works #OTHEREFR , we trained the model in an end-to-end manner by using a binary target.

Specifically, we assigned 1 to the n frames around the end-point of wake-up word and 0 to the remainder.

Here, the end-point was obtained by using a simple energy-based voice activity detector (VAD).

We decided to use n = 41 (i.e., 20 additional frames each before and after the end-point of the wake-up word) based on the development set result.

We trained the model during 5 epochs for distillation step and 50 epochs for fine-tuning step with the Adam optimizer #OTHEREFR . A batch size was set to 32 utterances.

We set the width multiplier α = 1/8 and an interpolation coefficient λ = 0.5. Pytorch framework #OTHEREFR was used for all experiments.

Lightweight Feature Encoder For Wake-Up Word Detection Based On Self-Supervised Speech Representation

Abstract: Self-supervised learning method that provides generalized speech representations has recently received increasing attention. Wav2vec 2.0 is the most famous example, showing remarkable performance in numerous downstream speech processing tasks. Despite its success, it is challenging to use it directly for wake-up word detection on mobile devices due to its expensive computational cost. In this work, we propose LiteFEW, a lightweight feature encoder for wake-up word detection that preserves the inherent ability of wav2vec 2.0 with a minimum scale. In the method, the knowledge of the pre-trained wav2vec 2.0 is compressed by introducing an autoencoder-based dimensionality reduction technique and distilled to LiteFEW. Experimental results on the open-source "Hey Snips" dataset show that the proposed method applied to various model structures significantly improves the performance, achieving over 20% of relative improvements with only 64k parameters.

Sgdr: Stochastic Gradient Descent With Warm Restarts

Abstract: Restart techniques are common in gradient-free optimization to deal with multimodal functions. Partial warm restarts are also gaining popularity in gradient-based optimization to improve the rate of convergence in accelerated gradient schemes to deal with ill-conditioned functions. In this paper, we propose a simple warm restart technique for stochastic gradient descent to improve its anytime performance when training deep neural networks. We empirically study its performance on the CIFAR-10 and CIFAR-100 datasets, where we demonstrate new state-of-the-art results at 3.14% and 16.21%, respectively. We also demonstrate its advantages on a dataset of EEG recordings and on a downsampled version of the ImageNet dataset. Our source code is available at https://github.com/loshchil/SGDR


In the choice of attack targets, we picked some representative commands as attack targets, #REFR .

To evaluate the transferability of our proposed combined adversarial attack strategy, we conducted experiments on commercial speech recognition APIs, such as those provided by iFlytek, Alibaba, and Baidu.

These APIs provide high-level English speech recognition services that directly impact the user experience of millions of people.

Attacks against these commercial APIs are closer to real-world attack patterns and are more sophisticated.

In this context, higher transferability implies a greater potential for danger.

Also, as shown in the previous experiments, the carrier of AEs plays a significant role in transferability.

The generation and transferability of AEs are directly influenced by the choice of carrier. In CommanderSong #OTHEREFR , Cheng Yuxuan et al.

first used music segments as the carrier of AEs in attacks, arguing that music has the nature of common consumption, giving it native opportunities in attacks with its popularity and extensive reach.

Attacks on music segments are likely to raise public concern.

Towards The Transferable Audio Adversarial Attack Via Ensemble Methods

Abstract: In recent years, deep learning (DL) models have achieved significant progress in many domains, such as autonomous driving, facial recognition, and speech recognition. However, the vulnerability of deep learning models to adversarial attacks has raised serious concerns in the community because of their insufficient robustness and generalization. Also, transferable attacks have become a prominent method for black-box attacks. In this work, we explore the potential factors that impact adversarial examples (AEs) transferability in DL-based speech recognition. We also discuss the vulnerability of different DL systems and the irregular nature of decision boundaries. Our results show a remarkable difference in the transferability of AEs between speech and images, with the data relevance being low in images but opposite in speech recognition. Motivated by dropout-based ensemble approaches, we propose random gradient ensembles and dynamic gradient-weighted ensembles, and we evaluate the impact of ensembles on the transferability of AEs. The results show that the AEs created by both approaches are valid for transfer to the black box API.

Improving Transferability Of Adversarial Examples With Input Diversity

Abstract: Though CNNs have achieved the state-of-the-art performance on various vision tasks, they are vulnerable to adversarial examples --- crafted by adding human-imperceptible perturbations to clean images. However, most of the existing adversarial attacks only achieve relatively low success rates under the challenging black-box setting, where the attackers have no knowledge of the model structure and parameters. To this end, we propose to improve the transferability of adversarial examples by creating diverse input patterns. Instead of only using the original images to generate adversarial examples, our method applies random transformations to the input images at each iteration. Extensive experiments on ImageNet show that the proposed attack method can generate adversarial examples that transfer much better to different networks than existing baselines. By evaluating our method against top defense solutions and official baselines from NIPS 2017 adversarial competition, the enhanced attack reaches an average success rate of 73.0%, which outperforms the top-1 attack submission in the NIPS competition by a large margin of 6.6%. We hope that our proposed attack strategy can serve as a strong benchmark baseline for evaluating the robustness of networks to adversaries and the effectiveness of different defense methods in the future. Code is available at https://github.com/cihangxie/DI-2-FGSM.


ClearGrasp #REFR dataset includes both a highly realistic synthetic dataset and a real-world benchmark.

Transparent object reconstruction requires the ground truth of the reconstructed depth or 3D shape for evaluation.

Therefore, datasets with ground truth of depth or 3D shapes are required for model training and evaluation.

In the subsection, we thoroughly summarise datasets published since 2020 for transparent object reconstruction, regarding year, place of publication (Pub.), number of objects in the images (#Obj.), dataset size (#Imgs), devices, auto-collection ability and special features.

The synthetic dataset is rendered by using the ray-tracing Cycles rendering engine integrated into Blender, which can provide important effects for transparent objects, such as refraction and soft shadow.

To capture the depth of transparent objects in the real world, transparent objects are sprayed with rough stone textures that can reflect light evenly and lead to better depth estimates from RGB-D cameras.

It should be noted that Clear-Grasp is the first large-scale dataset including 50k synthetic images and 286 real images for the depth reconstruction of transparent objects.

OOD #OTHEREFR : Omniverse Object Dataset consists of 60k synthetic images of five transparent objects from ClearGrasp #OTHEREFR .

The Omniverse Platform and NVIDIA PhysX engine are used for rendering those images and getting natural poses of objects.

Robotic Perception Of Transparent Objects: A Review

Abstract: Fig. 1. Typical applications of transparent object perception. (a) Robot assistant [1](©[2021] IEEE); (b) Autonomous robot navigation [2]; (c) Laboratory automation [3]; (d) Waste sorting and recycling [4].

Cleargrasp: 3D Shape Estimation Of Transparent Objects For Manipulation

Abstract: Transparent objects are a common part of everyday life, yet they possess unique visual properties that make them incredibly difficult for standard 3D sensors to produce accurate depth estimates for. In many cases, they often appear as noisy or distorted approximations of the surfaces that lie behind them. To address these challenges, we present ClearGrasp -- a deep learning approach for estimating accurate 3D geometry of transparent objects from a single RGB-D image for robotic manipulation. Given a single RGB-D image of transparent objects, ClearGrasp uses deep convolutional networks to infer surface normals, masks of transparent surfaces, and occlusion boundaries. It then uses these outputs to refine the initial depth estimates for all transparent surfaces in the scene. To train and test ClearGrasp, we construct a large-scale synthetic dataset of over 50,000 RGB-D images, as well as a real-world test benchmark with 286 RGB-D images of transparent objects and their ground truth geometries. The experiments demonstrate that ClearGrasp is substantially better than monocular depth estimation baselines and is capable of generalizing to real-world images and novel objects. We also demonstrate that ClearGrasp can be applied out-of-the-box to improve grasping algorithms' performance on transparent objects. Code, data, and benchmarks will be released. Supplementary materials available on the project website: this https URL


The performance on nuScenes-C is improved as the performance on the "clean" nuScenes #REFR dataset. The relation of absolute performance is close to linear.

In this study, we conduct a comprehensive benchmarking analysis of 26 existing BEV detectors on the nuScenes-C dataset.

The main results of our experiments are presented in Tables 2 and 3.

Our findings indicate that all models exhibit varying degrees of performance declines on the corruption set.

We observed that Bright, which causes a much larger shift in pixel distribution than Motion Blur, resulted (b) mRR vs. NDS Figure 3 .

However, when considering the relative performance, the mRR metric is more randomly distributed without a clear trend to increase. in the smallest performance drop.

For most of the models, the resilience rate of Bright remains the highest.

We notice a strong correlation of the absolute performances between nuScenes-C and the "clean" dataset.

Specifically, BEV detectors that perform well on the standard dataset are also likely to perform better on the out-ofdistribution dataset, as illustrated in Figure 3 (a).

However, a closer examination of the results revealed a more complex situation.

Robobev: Towards Robust Bird'S Eye View Perception Under Corruptions

Abstract: The recent advances in camera-based bird's eye view (BEV) representation exhibit great potential for in-vehicle 3D perception. Despite the substantial progress achieved on standard benchmarks, the robustness of BEV algorithms has not been thoroughly examined, which is critical for safe operations. To bridge this gap, we introduce RoboBEV, a comprehensive benchmark suite that encompasses eight distinct corruptions, including Bright, Dark, Fog, Snow, Motion Blur, Color Quant, Camera Crash, and Frame Lost. Based on it, we undertake extensive evaluations across a wide range of BEV-based models to understand their resilience and reliability. Our findings indicate a strong correlation between absolute performance on in-distribution and out-of-distribution datasets. Nonetheless, there are considerable variations in relative performance across different approaches. Our experiments further demonstrate that pre-training and depth-free BEV transformation has the potential to enhance out-of-distribution robustness. Additionally, utilizing long and rich temporal information largely helps with robustness. Our findings provide valuable insights for designing future BEV models that can achieve both accuracy and robustness in real-world deployments. 1

Nuscenes: A Multimodal Dataset For Autonomous Driving

Abstract: Robust detection and tracking of objects is crucial for the deployment of autonomous vehicle technology. Image-based benchmark datasets have driven development in computer vision tasks such as object detection, tracking and segmentation of agents in the environment. Most autonomous vehicles, however, carry a combination of cameras and range sensors such as lidar and radar. As machine learning based methods for detection and tracking become more prevalent, there is a need to train and evaluate such methods on datasets containing range sensor data along with images. In this work we present nuTonomy scenes (nuScenes), the first published dataset to carry the full autonomous vehicle sensor suite: 6 cameras, 5 radars and 1 lidar, all with full 360 degree field of view. nuScenes comprises 1000 scenes, each 20s long and fully annotated with 3D bounding boxes for 23 classes and 8 attributes. It has 7x as many annotations and 100x as many images as the pioneering KITTI dataset. We define novel 3D detection and tracking metrics. We also provide careful dataset analysis as well as baselines for lidar and image based detection and tracking. Data, development kit and more information are available online at this http URL.


With global attention mechanism, TD-Net #REFR aggregated the features from different time stamps and replaced the deep model with several shallow models distributed across the timeline.

Noticing the lack of information from nonkeyframes, Li et al.

#OTHEREFR extracted shallow features from non-keyframes, and fused them into the propagated deep features by spatially variant convolution.

To compensate for the spatial misalignment between video frames, Zhu et al. #OTHEREFR and Xu et al.

#OTHEREFR warped the intermediate features from keyframes by optical flow to produce segmentation results for non-keyframes. Jain et al.

#OTHEREFR fused the shallow features of non-keyframe into the warped features, and decoded them into better results.

All the above methods mainly reduced the depth of backbone networks, but neglected the factor of input resolution considered in this paper.

Instead of processing the image frames as a whole, Verelst et al.

#OTHEREFR split the frame into blocks and chose to copy or process them by a policy network.

This block-based method reduces computational overhead from the spatial dimension, but lacks global information on nonkeyframes. Kim et al. #OTHEREFR attempted to improve efficiency by reducing resolution.

But they directly used the LR segmentation results, thus suffering from severe performance degradation.

Efficient Semantic Segmentation By Altering Resolutions For Compressed Videos

Abstract: Video semantic segmentation (VSS) is a computationally expensive task due to the per-frame prediction for videos of high frame rates. In recent work, compact models or adaptive network strategies have been proposed for efficient VSS. However, they did not consider a crucial factor that affects the computational cost from the input side: the input resolution. In this paper, we propose an altering resolution framework called AR-Seg for compressed videos to achieve efficient VSS. AR-Seg aims to reduce the computational cost by using low resolution for non-keyframes. To prevent the performance degradation caused by downsampling, we design a Cross Resolution Feature Fusion (CR-eFF) module, and supervise it with a novel Feature Similarity Training (FST) strategy. Specifically, CReFF first makes use of motion vectors stored in a compressed video to warp features from high-resolution keyframes to low-resolution non-keyframes for better spatial alignment, and then selectively aggregates the warped features with local attention mechanism. Furthermore, the proposed FST supervises the aggregated features with high-resolution features through an explicit similarity loss and an implicit constraint from the shared decoding layer. Extensive experiments on CamVid and Cityscapes show that AR-Seg achieves state-of-the-art performance and is compatible with different segmentation backbones. On CamVid, AR-Seg saves 67% computational cost (measured in GFLOPs) with the PSPNet18 backbone while maintaining high segmentation accuracy.

Temporally Distributed Networks For Fast Video Semantic Segmentation

Abstract: We present TDNet, a temporally distributed network designed for fast and accurate video semantic segmentation. We observe that features extracted from a certain high-level layer of a deep CNN can be approximated by composing features extracted from several shallower sub-networks. Leveraging the inherent temporal continuity in videos, we distribute these sub-networks over sequential frames. Therefore, at each time step, we only need to perform a lightweight computation to extract a sub-features group from a single sub-network. The full features used for segmentation are then recomposed by application of a novel attention propagation module that compensates for geometry deformation between frames. A grouped knowledge distillation loss is also introduced to further improve the representation power at both full and sub-feature levels. Experiments on Cityscapes, CamVid, and NYUD-v2 demonstrate that our method achieves state-of-the-art accuracy with significantly faster speed and lower latency.


Bayesian optimization is a global optimization scheme that has been commonly used to optimize neural network training parameters #REFR .

Other methods have utilized global optimization techniques such as branch and bound to determine the optimal transform in certain situations such as scan to model matching #OTHEREFR .

In situations where outliers, dynamic obstacles, or noisy sensor measurements exist, these global methods cannot guarantee an optimal solution #OTHEREFR .

The lack of consistently reliable methods for global registration motivates research targeted at more robust initialization techniques.

In this paper, we outline a framework based on Bayesian optimization (BO) #OTHEREFR , a global optimization method, to systematically compute T 0 , and find this approach produces more accurate alignments than state-of-the-art methods.

It is fundamentally compatible with variants of ICP that address point correspondences, the weighting of correspondences, and other methods that focus on adjusting how the objective is constructed.

In practice, BO aims to find the minima or maxima of an objective function and is effective in situations where the objective is either complex, noisy, or expensive to calculate, such as in the case of minimizing the point-topoint correspondence distance.

BO utilizes a computationally efficient probabilistic model of the objective function (see Figure 1 ) that is inexpensive to evaluate.

As the point cloud registration problem as formulated by ICP is inherently a non-convex problem that requires expensive iterations for each initial T 0 estimate, it is an ideal candidate for BO.

We demonstrate that our approach outperforms exhaustive searches for this initial estimate, as well as other "globally optimal" methods.

In this work, we present an open-source Bayesian optimization-based method (BO-ICP) for determining the crucial initial condition for ICP problems.

Bo-Icp: Initialization Of Iterative Closest Point Based On Bayesian Optimization

Abstract: Typical algorithms for point cloud registration such as Iterative Closest Point (ICP) require a favorable initial transform estimate between two point clouds in order to perform a successful registration. State-of-the-art methods for choosing this starting condition rely on stochastic sampling or global optimization techniques such as branch and bound. In this work, we present a new method based on Bayesian optimization for finding the critical initial ICP transform. We provide three different configurations for our method which highlights the versatility of the algorithm to both find rapid results and refine them in situations where more runtime is available such as offline map building. Experiments are run on popular data sets and we show that our approach outperforms state-of-the-art methods when given similar computation time. Furthermore, it is compatible with other improvements to ICP, as it focuses solely on the selection of an initial transform, a starting point for all ICP-based methods.

Practical Bayesian Optimization Of Machine Learning Algorithms

Abstract: The use of machine learning algorithms frequently involves careful tuning of learning parameters and model hyperparameters. Unfortunately, this tuning is often a "black art" requiring expert experience, rules of thumb, or sometimes brute-force search. There is therefore great appeal for automatic approaches that can optimize the performance of any given learning algorithm to the problem at hand. In this work, we consider this problem through the framework of Bayesian optimization, in which a learning algorithm's generalization performance is modeled as a sample from a Gaussian process (GP). We show that certain choices for the nature of the GP, such as the type of kernel and the treatment of its hyperparameters, can play a crucial role in obtaining a good optimizer that can achieve expertlevel performance. We describe new algorithms that take into account the variable cost (duration) of learning algorithm experiments and that can leverage the presence of multiple cores for parallel experimentation. We show that these proposed algorithms improve on previous automatic procedures and can reach or surpass human expert-level optimization for many algorithms including latent Dirichlet allocation, structured SVMs and convolutional neural networks.


A T E X I T S H A 1 _ B A S E 6 4 = " Z U D 8 A P B L U T 3 F R + 8 H X N L Y U 7 A S S 3 8 = " > A

Therefore, designing real-time FC search algorithms critically hinges on finding an accurate and efficient solution to Q in #REFR .

These estimates are widely referred to as Q-values, denoted by

where a higher Q(S i , a i ) indicates that taking an action a i from a MDP state S i yields better aggregate future rewards #OTHEREFR .

Note that, however, when solving Q in a POMDP environment, the ordinary Q-learning algorithm is ineffective as partial observations O i are typically not reflective of the underlying POMDP state S i .

As a result, Q(O i , a i ) = Q(S i , a i ).

To this end, we aim to design an approach that judiciously exploits the underlying structure of each POMDP state S i .

In particular, we develop a graph recurrent Q-network (GRQN) architecture that exploits the following three key elements:

Grnn-Based Real-Time Fault Chain Prediction

Abstract: This paper proposes a data-driven graphical framework for the real-time search of risky cascading fault chains (FCs). While identifying risky FCs is pivotal to alleviating cascading failures, the complex spatio-temporal dependencies among the components of the power system render challenges to modeling and analyzing FCs. Furthermore, the real-time search of risky FCs faces an inherent combinatorial complexity that grows exponentially with the size of the system. The proposed framework leverages the recent advances in graph recurrent neural networks to circumvent the computational complexities of the real-time search of FCs. The search process is formalized as a partially observable Markov decision process (POMDP), which is subsequently solved via a time-varying graph recurrent neural network (GRNN) that judiciously accounts for the inherent temporal and spatial structures of the data generated by the system. The key features of this structure include (i) leveraging the spatial structure of the data induced by the system topology, (ii) leveraging the temporal structure of data induced by system dynamics, and (iii) efficiently summarizing the system's history in the latent space of the GRNN. The proposed framework's efficiency is compared to the relevant literature on the IEEE 39-bus New England system and the IEEE 118-bus system.

Risk Assessment Of Multi-Timescale Cascading Outages Based On Markovian Tree Search

Abstract: In the risk assessment of cascading outages, the rationality of simulation and efficiency of computation are both of great significance. To overcome the drawback of sampling-based methods that huge computation resources are required and the shortcoming of initial contingency selection practices that the dependencies in sequences of outages are omitted, this paper proposes a novel risk assessment approach by searching on Markovian Tree. The Markovian tree model is reformulated from the quasi-dynamic multitimescale simulation model proposed recently to ensure reasonable modeling and simulation of cascading outages. Then, a tree search scheme is established to avoid duplicated simulations on same cascade paths, significantly saving the computation time. To accelerate the convergence of a risk assessment, a risk estimation index is proposed to guide the search for states with major contributions to the risk, and the risk assessment is realized based on the risk estimation index with a forward tree search and backward update algorithm. The effectiveness of the proposed method is illustrated on a four-node power system, and its convergence profile as well as efficiency is demonstrated on the RTS-96 test system.


Similar to the loss functions, the final cost includes three items, a classification cost C cls , a box L1 cost C bbox , and a GIOU cost C GIOU #REFR . We focus only on the classification cost as well.

Typically, a ground truth will be assigned only one prediction as the positive example.

Predictions with no ground truths assigned will be viewed as negative examples.

To assign predictions with ground truths, we first calculate a cost matrix C ∈ R N pred ×Ngt between them.

The N pred and N gt are the number for predictions and ground truths.

Then a Hungarian matching algorithm will perform on the cost matrix to assign each ground truth a prediction by minimizing sum costs.

For the i th prediction and the j th ground truth, the classification cost is:

C cls (i, j) = |1 − p i | γ BCE(p i , 1)−p γ i BCE(1−p i , 1). (2)

The formulation is similar to the focal cost but has a litter modification #OTHEREFR .

The focal loss only encourages positive examples to predict 1, while the classification cost adds an additional penalty term to avoid it to 0.

Detection Transformer With Stable Matching

Abstract: This paper is concerned with the matching stability problem across different decoder layers in DEtection TRansformers (DETR). We point out that the unstable matching in DETR is caused by a multi-optimization path problem, which is highlighted by the one-to-one matching design in DETR. To address this problem, we show that the most important design is to use and only use positional metrics (like IOU) to supervise classification scores of positive examples. Under the principle, we propose two simple yet effective modifications by integrating positional metrics to DETR's classification loss and matching cost, named positionsupervised loss and position-modulated cost. We verify our methods on several DETR variants. Our methods show consistent improvements over baselines. By integrating our methods with DINO, we achieve 50.4 and 51.5 AP on the COCO detection benchmark using ResNet-50 backbones under 1× (12 epochs) and 2× (24 epochs) training settings, achieving a new record under the same setting. We achieve 63.8 AP on COCO detection test-dev with a Swin-Large backbone. Our code will be made available at https:// github.com/IDEA-Research/Stable-DINO. * Equal contributions. List order is random. † This work was done when Shilong Liu, Hao Zhang, Feng Li, and Hongyang Li were interns at IDEA.

Generalized Intersection Over Union: A Metric And A Loss For Bounding Box Regression

Abstract: Intersection over Union (IoU) is the most popular evaluation metric used in the object detection benchmarks. However, there is a gap between optimizing the commonly used distance losses for regressing the parameters of a bounding box and maximizing this metric value. The optimal objective for a metric is the metric itself. In the case of axis-aligned 2D bounding boxes, it can be shown that IoU can be directly used as a regression loss. However, IoU has a plateau making it infeasible to optimize in the case of non-overlapping bounding boxes. In this paper, we address the this weakness by introducing a generalized version of IoU as both a new loss and a new metric. By incorporating this generalized IoU ( GIoU) as a loss into the state-of-the art object detection frameworks, we show a consistent improvement on their performance using both the standard, IoU based, and new, GIoU based, performance measures on popular object detection benchmarks such as PASCAL VOC and MS COCO.


We use the train/val/test partitions introduced in Anderson et al. #REFR (train/val/test: 100K/18K/13K samples) for evaluation.

Dataset QM9 #OTHEREFR is a dataset of 130k stable and synthetically accessible organic molecules with up to 9 heavy atoms (29 atoms including hydrogens).

In this section, we train diffusion models to generate atoms' (1) 3-dimensional coordinates; (2) types (H, C, N, O, F) and 3integer-valued atom charges.

Evaluation Metrics Our goal here is to generate molecules targeting some desired properties while at the same time not harming general generation quality (e.g., molecules' validity (the proportion of atoms with right valency) and stability, etc.).

In such a scenario, a molecule is represented as a point cloud, in which each point denotes a single atom and has its own (atom) type.

Following #OTHEREFR , for each pair of atoms, we use the distance between them and the atoms' types to predict bonds (single, double, triple, or none) between atoms.

In this section, we consider optimizing two desired properties: (1) quantitative estimate of druglikeness (QED) #OTHEREFR (how likely a molecule is a potential drug candidate based on marketed drug molecules) and (2) synthetic accessibility score (SA) (the difficulty of drug synthesis), which are crucial in drug discovery domain.

A good method should have a high averaged QED and SA.

Towards Controllable Diffusion Models Via Reward-Guided Exploration

Abstract: By formulating data samples' formation as a Markov denoising process, diffusion models achieve state-of-the-art performances in a collection of tasks. Recently, many variants of diffusion models have been proposed to enable controlled sample generation. Most of these existing methods either formulate the controlling information as an input (i.e.,: conditional representation) for the noise approximator, or introduce a pre-trained classifier in the test-phase to guide the Langevin dynamic towards the conditional goal. However, the former line of methods only work when the controlling information can be formulated as conditional representations, while the latter requires the pre-trained guidance classifier to be differentiable. In this paper, we propose a novel framework named RGDM (Reward-Guided Diffusion Model) that guides the training-phase of diffusion models via reinforcement learning (RL). The proposed training framework bridges the objective of weighted log-likelihood and maximum entropy RL, which enables calculating policy gradients via samples from a pay-off distribution proportional to exponential scaled rewards, rather than from policies themselves. Such a framework alleviates the high gradient variances and enables diffusion models to explore for highly rewarded samples in the reverse process. Experiments on 3D shape and molecule generation tasks show significant improvements over existing conditional diffusion models.

Cormorant: Covariant Molecular Neural Networks

Abstract: We propose Cormorant, a rotationally covariant neural network architecture for learning the behavior and properties of complex many-body physical systems. We apply these networks to molecular systems with two goals: learning atomic potential energy surfaces for use in Molecular Dynamics simulations, and learning ground state properties of molecules calculated by Density Functional Theory. Some of the key features of our network are that (a) each neuron explicitly corresponds to a subset of atoms; (b) the activation of each neuron is covariant to rotations, ensuring that overall the network is fully rotationally invariant. Furthermore, the non-linearity in our network is based upon tensor products and the Clebsch-Gordan decomposition, allowing the network to operate entirely in Fourier space. Cormorant significantly outperforms competing algorithms in learning molecular Potential Energy Surfaces from conformational geometries in the MD-17 dataset, and is competitive with other methods at learning geometric, energetic, electronic, and thermodynamic properties of molecules on the GDB-9 dataset.


This implies J ν (h ν (g), h ν (s)) ≥ α which is equivalent to #REFR .

To show (1), let α be the right-hand side of this inequality. This means that

α = min r∈Rs c n(s)−n(r) J ν (h ν (g), h ν (r)).

If α = 0 then (1) is trivially true. Otherwise, put m = h ν (g)/α.

Due to the bilinearity of J ν , vector m is a feasible solution to P .

If the objective value of P is at least 1, then (again by bilinearity) we have in particular J ν (h ν (g)/α, h ν (s)) ≥ 1.

Note that if the objective value of the linear program is less than one but still more than zero then exponential bound for a smaller c might hold.

This theorem also holds for any other linear representation which maps gadgets to non-negative vectors.

The downside of a linear program is that it is usually solved by a numerical method which is not suitable for a theoretical proof. We circumvent this by solving the dual problem.

Note that any solution of the dual gives us a lower bound but of course suboptimal solutions will give weaker bounds.

So for a given solution of the dual we only need to certify that it is indeed a solution which is easy and we do not need to prove optimality.

Counting Circuit Double Covers

Abstract: We study a counting version of Cycle Double Cover Conjecture. We discuss why it is more interesting to count circuits (i.e., graphs isomorphic to C k for some k) instead of cycles (graphs with all degrees even). We give an almost-exponential lower-bound for graphs with a surface embedding of representativity at least 4. We also prove an exponential lower-bound for planar graphs. We conjecture that any bridgeless cubic graph has at least 2 n/2−1 circuit double covers and we show an infinite class of graphs for which this bound is tight.

A Unified Approach To Distance-Two Colouring Of Graphs On Surfaces

Abstract: In this paper we introduce the notion of $\Sigma$-colouring of a graph $G$: For given subsets $\Sigma(v)$ of neighbours of $v$, for every $v\in V(G)$, this is a proper colouring of the vertices of $G$ such that, in addition, vertices that appear together in some $\Sigma(v)$ receive different colours. This concept generalises the notion of colouring the square of graphs and of cyclic colouring of graphs embedded in a surface. We prove a general result for graphs embeddable in a fixed surface, which implies asymptotic versions of Wegner's and Borodin's Conjecture on the planar version of these two colourings. Using a recent approach of Havet et al., we reduce the problem to edge-colouring of multigraphs, and then use Kahn's result that the list chromatic index is close to the fractional chromatic index. Our results are based on a strong structural lemma for graphs embeddable in a fixed surface, which also implies that the size of a clique in the square of a graph of maximum degree $\Delta$ embeddable in some fixed surface is at most $\frac32\,\Delta$ plus a constant.